-- Logs begin at Tue 2022-08-30 01:16:55 UTC, end at Tue 2022-08-30 02:16:47 UTC. -- Aug 30 01:41:37.801077 dv-cinder-819668-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 01:41:37.815061 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 01:41:37.817655 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 01:41:37 2022] *** Aug 30 01:41:37.817795 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 01:41:37.817915 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 01:41:37.818020 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: nodename: dv-cinder-819668-6-iscsi Aug 30 01:41:37.818126 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: machine: x86_64 Aug 30 01:41:37.818230 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: clock source: unix Aug 30 01:41:37.818330 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: pcre jit disabled Aug 30 01:41:37.818463 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: detected number of CPU cores: 4 Aug 30 01:41:37.818574 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: current working directory: / Aug 30 01:41:37.818669 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: detected binary path: /usr/bin/uwsgi-core Aug 30 01:41:37.818767 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: your processes number limit is 47680 Aug 30 01:41:37.818862 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: your memory page size is 4096 bytes Aug 30 01:41:37.818962 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: detected max file descriptor number: 2048 Aug 30 01:41:37.819071 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: lock engine: pthread robust mutexes Aug 30 01:41:37.819324 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: thunder lock: enabled Aug 30 01:41:37.819604 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 01:41:37.819788 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 01:41:37.859522 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: Python main interpreter initialized at 0x55cb5fd879c0 Aug 30 01:41:37.859730 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: python threads support enabled Aug 30 01:41:37.859868 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: your server socket listen backlog is limited to 100 connections Aug 30 01:41:37.860059 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: your mercy for graceful operations on workers is 90 seconds Aug 30 01:41:37.860792 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 01:41:37.860922 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: *** Operational MODE: preforking *** Aug 30 01:41:37.861178 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: *** uWSGI is running in multiple interpreter mode *** Aug 30 01:41:37.861178 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: spawned uWSGI master process (pid: 81415) Aug 30 01:41:37.861395 dv-cinder-819668-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 01:41:37.865381 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: spawned uWSGI worker 1 (pid: 81428, cores: 1) Aug 30 01:41:37.865702 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: spawned uWSGI worker 2 (pid: 81429, cores: 1) Aug 30 01:41:37.866762 dv-cinder-819668-6-iscsi devstack@keystone.service[81415]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 01:41:42.109238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:41:42.109238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: from cryptography.utils import int_from_bytes Aug 30 01:41:42.110015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:41:42.110015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: from cryptography.utils import int_from_bytes Aug 30 01:41:42.110692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:41:42.110692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: from cryptography.utils import int_from_bytes Aug 30 01:41:42.110799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 01:41:42.110799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: from cryptography.utils import int_from_bytes Aug 30 01:41:42.159340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 01:41:42.159544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 01:41:42.159711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] command line args: [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 01:41:42.159900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 01:41:42.159997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 01:41:42.160104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 01:41:42.160232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ================================================================================ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 01:41:42.160356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] command line args: [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 01:41:42.160429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] admin_token = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.160580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 01:41:42.160671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_dir = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.160738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ================================================================================ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 01:41:42.160808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.160944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_source = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] admin_token = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] debug = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_dir = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_source = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] debug = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.161979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_config_append = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.162898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_dir = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_config_append = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_dir = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.163995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.164974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.165970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.166134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.166221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.166369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.166453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.166617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.166696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.166862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] publish_errors = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] publish_errors = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.167910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] standard_threads = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.168922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] standard_threads = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_journal = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_json = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_stderr = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_syslog = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.169985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_journal = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.170061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.170132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_json = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.170260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_stderr = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.170345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.170422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_syslog = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.170532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.170651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 01:41:42.170785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.170903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.171968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.172953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173585 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.173903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.174857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.175958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.176035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.176185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.177778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.177778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.178948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.179873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.180866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.181990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.182920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.183995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.184925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.185938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.186906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.187942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.188999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189459 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.189934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.190870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.191970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.192047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.192208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.192314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.192690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.192766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.192836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.192904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193427 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.193861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.194945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.external = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.195950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.196641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.196732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.196898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.196983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.password = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.token = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.external = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.197922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.password = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.token = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.198875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.199960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.200995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.201950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.202920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203547 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.203873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.204835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205239 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.205945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.206969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.207896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.208858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.209984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210803 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.210958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.211928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212670 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.212897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.213977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.214931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.215963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.216907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.217067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.217187 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.217830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.217909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.218995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.219940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.220905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.221860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.222927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.223948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.224900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225372 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.225978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.226928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.227980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.228970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.229903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.230845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231243 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.231943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.232923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.233880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.234851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.235020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.235098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.235253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.235666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.235745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.235903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.235979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.236984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237214 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.237941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238623 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.238852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.239960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240459 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.240933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.241972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.242911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.243893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.244857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.245955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.246963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.247115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.247192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.247272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 01:41:42.247345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.247580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.247817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.248062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.248313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.248554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.248789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 01:41:42.248964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 01:41:42.276849 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.277075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.277297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.278146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.278380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.278606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.285482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.286852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.313110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.313336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.314399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.314630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.366774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.367117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.415202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.415431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.416031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.416284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.569307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.569530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.569755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.569863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.569970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.570097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.570179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.570321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.570402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.570541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.570756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.570985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.593389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.593626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.593719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.593844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.593927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.594917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.605410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.605951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.618882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.619922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.620004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.620133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.620212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.620377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 01:41:42.637308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.637514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:41:42.637814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.637896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.638024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:41:42.638120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:41:42.638299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.638379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.638548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 01:41:42.638628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.638801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.638883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.639054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.639313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.639580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.639840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.648532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.648805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.649071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.649150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.649373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.649467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.649688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.649967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.656275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.656969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.659142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.659423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.659704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.659833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.660018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.660103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.660266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.660410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.660530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.660694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.660794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.660978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.661060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.661239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.661317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.661508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.661595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.661781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.661860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.662163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.662441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.662710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.684743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.685613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.686188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.686469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.687066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.687347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.690971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.691252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.691527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.691966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.692267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.692562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.698582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.698871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.699429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.699717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.702887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.703177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.703465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.703754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.703834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.704027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.704109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.704345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.704631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.704961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.705253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.705524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.705791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.705950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.706237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.706521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.722024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.722300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.722759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.723036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.726153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.726914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.729274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.729567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.730163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.730461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.733886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.734661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.737278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.738063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.740367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.740665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.740942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.741219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.741299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.741492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.741580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.741861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.741942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.742145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.742422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.742702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.758365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.759252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.759525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.759669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.759796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.760144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.760587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.760992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.766490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.768121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.769693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.769833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.769970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.771584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.772605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.772890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.773170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.773458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.773539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.773750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.773860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.776572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.776858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.777267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.777560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.777847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.784994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.785286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.785566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.785849 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.786127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.786401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.786684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.789013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.789300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.789677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.790488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.790767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.791040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.791313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.802505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.806061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.807261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.808933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.810174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.810372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.811822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.812102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.812401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.813088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.814521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.816109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.816437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.816721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.818896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.820384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.823264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.823554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.823756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.823837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.824091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.825245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.828226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.828542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.828822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.829104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.831961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.832261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.832554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.836916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.837208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.837489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.839483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.839771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.840049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.840350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.840621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.840889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.841170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.841442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.841721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.841991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.842258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.842514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.842771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.843034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.844389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 01:41:42.844671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.844950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.845221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.845493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.845775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.846058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.846333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.846611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.846890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.847159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.847417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.847694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.847950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 01:41:42.877884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cb5fd879c0 pid: 81429 (default app) Aug 30 01:41:42.881222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8217e3-155c-47c3-95f6-f2d710a80e50 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:42.881506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8217e3-155c-47c3-95f6-f2d710a80e50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:42.881778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8217e3-155c-47c3-95f6-f2d710a80e50 None None] PATH_INFO: `/v3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:42.882994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cb5fd879c0 pid: 81428 (default app) Aug 30 01:41:42.885944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Tue Aug 30 01:41:42 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:41:49.351597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e9c8f-0aff-42c5-a141-0096c1a008f0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:49.352040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e9c8f-0aff-42c5-a141-0096c1a008f0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:49.352144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e9c8f-0aff-42c5-a141-0096c1a008f0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:49.356421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:41:49.361677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:49.362105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:49.362237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:49.431952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: INFO dbcounter [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] Registered counter for database keystone Aug 30 01:41:49.451037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG oslo_db.sqlalchemy.engines [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81429) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 01:41:49.457345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writer thread running {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 01:41:49.638495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG passlib.handlers.bcrypt [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81429) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 01:41:49.638805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG passlib.handlers.bcrypt [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81429) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 01:41:49.696969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:49.698559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:49.825947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 01:41:50.266830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e9c8f-0aff-42c5-a141-0096c1a008f0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:50.336319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: INFO dbcounter [None req-5f5e9c8f-0aff-42c5-a141-0096c1a008f0 None None] Registered counter for database keystone Aug 30 01:41:50.357745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG oslo_db.sqlalchemy.engines [None req-5f5e9c8f-0aff-42c5-a141-0096c1a008f0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 01:41:50.363975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writer thread running {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 01:41:50.480310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e9c8f-0aff-42c5-a141-0096c1a008f0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:50.484488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4bf7-2fbc-411f-9d00-f3ff38838955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:50.485100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4bf7-2fbc-411f-9d00-f3ff38838955 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:50.485621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4bf7-2fbc-411f-9d00-f3ff38838955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:50.485899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4bf7-2fbc-411f-9d00-f3ff38838955 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:50.493991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4bf7-2fbc-411f-9d00-f3ff38838955 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:50.545302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9efe4bf7-2fbc-411f-9d00-f3ff38838955 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 01:41:50.545586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4bf7-2fbc-411f-9d00-f3ff38838955 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:50.554738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Tue Aug 30 01:41:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:41:50.560720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:50.575335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e4b99-399d-46b2-a734-543fcc31aa05 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:50.579738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c2141-e0f1-41bd-8051-a5f2a880aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:50.580858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c2141-e0f1-41bd-8051-a5f2a880aee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:50.581138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c2141-e0f1-41bd-8051-a5f2a880aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:50.581408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c2141-e0f1-41bd-8051-a5f2a880aee7 admin admin] PATH_INFO: `/v3/projects/e7c73461b5344be681351a70b1beafa9` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:50.582961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c2141-e0f1-41bd-8051-a5f2a880aee7 admin admin] RBAC: Authorizing `identity:get_project(project_id=e7c73461b5344be681351a70b1beafa9)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:50.635488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING keystone.common.rbac_enforcer.enforcer [None req-818c2141-e0f1-41bd-8051-a5f2a880aee7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 01:41:50.635840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c2141-e0f1-41bd-8051-a5f2a880aee7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:50.640361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Tue Aug 30 01:41:50 2022] GET /identity/v3/projects/e7c73461b5344be681351a70b1beafa9 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:41:52.537213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa705-9b33-4842-9b49-0bc6e3a94d9f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:52.537734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa705-9b33-4842-9b49-0bc6e3a94d9f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:52.537734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaa705-9b33-4842-9b49-0bc6e3a94d9f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:52.542277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:41:52.547823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc61b34-34e6-45fa-bab4-c9deb20d970b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:52.548225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc61b34-34e6-45fa-bab4-c9deb20d970b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:52.548334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc61b34-34e6-45fa-bab4-c9deb20d970b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:52.568515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cbc61b34-34e6-45fa-bab4-c9deb20d970b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:52.569988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cbc61b34-34e6-45fa-bab4-c9deb20d970b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:52.577610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 30 01:41:52.843702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaa705-9b33-4842-9b49-0bc6e3a94d9f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:52.855474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaa705-9b33-4842-9b49-0bc6e3a94d9f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:52.859577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ecb8d-6e50-4581-9e5a-809fc54e7689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:52.860170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ecb8d-6e50-4581-9e5a-809fc54e7689 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:52.860730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ecb8d-6e50-4581-9e5a-809fc54e7689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:52.860999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ecb8d-6e50-4581-9e5a-809fc54e7689 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:52.868631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ecb8d-6e50-4581-9e5a-809fc54e7689 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:52.870370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ecb8d-6e50-4581-9e5a-809fc54e7689 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:52.878875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Tue Aug 30 01:41:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:41:52.884466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc61b34-34e6-45fa-bab4-c9deb20d970b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:52.893393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc61b34-34e6-45fa-bab4-c9deb20d970b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:52.897740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66202df-a541-4ca0-a362-4f19d81c0663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:52.898357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66202df-a541-4ca0-a362-4f19d81c0663 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:52.898632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66202df-a541-4ca0-a362-4f19d81c0663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:52.898902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66202df-a541-4ca0-a362-4f19d81c0663 admin admin] PATH_INFO: `/v3/users/a8eb78c86bca426c9163deb9994b5cec` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:52.900832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66202df-a541-4ca0-a362-4f19d81c0663 admin admin] RBAC: Authorizing `identity:get_user(user_id=a8eb78c86bca426c9163deb9994b5cec)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:52.902651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66202df-a541-4ca0-a362-4f19d81c0663 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:52.904152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Tue Aug 30 01:41:52 2022] GET /identity/v3/users/a8eb78c86bca426c9163deb9994b5cec => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:41:54.839739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf7e53-a5ff-46ca-a71d-c328a66d96eb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:54.840352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf7e53-a5ff-46ca-a71d-c328a66d96eb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:54.840352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf7e53-a5ff-46ca-a71d-c328a66d96eb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:54.841726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:41:54.846999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2674d-0330-4b2f-b191-ca662894c74a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:54.847400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2674d-0330-4b2f-b191-ca662894c74a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:54.847749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2674d-0330-4b2f-b191-ca662894c74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:54.866474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-53b2674d-0330-4b2f-b191-ca662894c74a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:54.868274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-53b2674d-0330-4b2f-b191-ca662894c74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:54.893648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5ed55-8008-4a40-8350-881dbeaf594b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:54.893933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5ed55-8008-4a40-8350-881dbeaf594b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:54.894208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5ed55-8008-4a40-8350-881dbeaf594b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:54.895502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:41:54.900740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:54.901039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:54.901308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:54.943020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG passlib.handlers.bcrypt [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81428) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 01:41:54.943408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG passlib.handlers.bcrypt [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81428) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 01:41:54.947464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:41:54.992437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:54.994519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:55.091926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:41:55.209771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2674d-0330-4b2f-b191-ca662894c74a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.234187 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:55.315652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2674d-0330-4b2f-b191-ca662894c74a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.348577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.349146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.349425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.349702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.361279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:41:55.363771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:41:55.370228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:41:55.371220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:41:55.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:41:55.376976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.456506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:55.461224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc42aa2-da30-4f8d-ba2c-85febedbab74 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.464374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:55.475705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d27826-2d3d-41e1-a513-a6ffa7eedc2d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.498677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.499542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.499830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.500105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.510360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.511315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.511597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.511872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.512864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:41:55.513167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:55.514552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:55.526950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 8/15] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:41:55.532604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.576551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:55.578070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:55.630863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3c06-4e39-4f6c-8837-25d88cb9b7de None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.664361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.665185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.665460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.665736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] PATH_INFO: `/v3/users/a8eb78c86bca426c9163deb9994b5cec` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.666852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] RBAC: Authorizing `identity:get_user(user_id=a8eb78c86bca426c9163deb9994b5cec)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:55.668310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:55.669852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 9/16] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/users/a8eb78c86bca426c9163deb9994b5cec => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:41:55.672660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 8/17] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:41:55.675325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.678118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.681227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:55.689652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:55.773063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd4679-0101-45ed-821c-09015a644643 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.774762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd769e-1337-40c9-a04a-da775f5079d5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.806517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.806876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.807133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.807408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.807673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.807790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.808046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.808355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.808836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:55.810257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:55.811774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 10/18] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:41:55.817646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.827958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:55.829504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:41:55.848670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:41:55.850207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:41:55.851181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:41:55.851181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:41:55.851181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:41:55.851181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 9/19] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:41:55.855556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:55.904184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:55.914323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:55.923866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694e959-03d2-4b8a-9204-5b4497474cbd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.957489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0e145-d3ba-43cd-af56-a69b655e29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.958092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e145-d3ba-43cd-af56-a69b655e29c1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.958364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e145-d3ba-43cd-af56-a69b655e29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.958630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e145-d3ba-43cd-af56-a69b655e29c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.959344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0e145-d3ba-43cd-af56-a69b655e29c1 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.domain.id=default, user.id=a8eb78c86bca426c9163deb9994b5cec) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:41:55.959643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0e145-d3ba-43cd-af56-a69b655e29c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:55.961156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0e145-d3ba-43cd-af56-a69b655e29c1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:55.961898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f1c5f-1884-4f93-b5c2-b57cac2b9f56 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:55.967514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 11/20] 10.222.0.50 () {56 vars in 1322 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/role_assignments?user.id=a8eb78c86bca426c9163deb9994b5cec&scope.domain.id=default&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:41:55.996833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1f25-acdd-443b-b8e1-3687d95d11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:55.997458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1f25-acdd-443b-b8e1-3687d95d11fb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:55.997796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1f25-acdd-443b-b8e1-3687d95d11fb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:55.998078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1f25-acdd-443b-b8e1-3687d95d11fb None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:55.998742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1f25-acdd-443b-b8e1-3687d95d11fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:41:55.999045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1f25-acdd-443b-b8e1-3687d95d11fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:56.000498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1f25-acdd-443b-b8e1-3687d95d11fb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:56.020857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 10/21] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:41:58.002515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469225fb-6f8c-4df5-9504-cc236ebf29a0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.003046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469225fb-6f8c-4df5-9504-cc236ebf29a0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.003046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469225fb-6f8c-4df5-9504-cc236ebf29a0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.004759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 12/22] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:41:58.010717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891c296-6601-4233-bfbf-968e5c2615bb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.011187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891c296-6601-4233-bfbf-968e5c2615bb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.011275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891c296-6601-4233-bfbf-968e5c2615bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.045323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9891c296-6601-4233-bfbf-968e5c2615bb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:58.046550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9891c296-6601-4233-bfbf-968e5c2615bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:58.122743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 11/23] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:41:58.330064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d269-d156-4ae4-9327-642208e85381 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.330343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d269-d156-4ae4-9327-642208e85381 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.330611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d269-d156-4ae4-9327-642208e85381 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.332226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 13/24] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:41:58.337829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d11c5d-dddc-444b-a7e0-a558a47f77d3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.338163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d11c5d-dddc-444b-a7e0-a558a47f77d3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.338446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d11c5d-dddc-444b-a7e0-a558a47f77d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.351160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10287c2-dfd3-4149-b507-99fae33f81b0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.351443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10287c2-dfd3-4149-b507-99fae33f81b0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.351712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10287c2-dfd3-4149-b507-99fae33f81b0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.353825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 14/25] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:41:58.358652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7477b7-a16b-4d60-bece-682e122b1209 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.358937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7477b7-a16b-4d60-bece-682e122b1209 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.359259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7477b7-a16b-4d60-bece-682e122b1209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.365355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e4d11c5d-dddc-444b-a7e0-a558a47f77d3 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:58.366557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e4d11c5d-dddc-444b-a7e0-a558a47f77d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:58.401261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fe7477b7-a16b-4d60-bece-682e122b1209 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:58.402869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fe7477b7-a16b-4d60-bece-682e122b1209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:58.428516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:58.481811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 12/26] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:41:58.482730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d11c5d-dddc-444b-a7e0-a558a47f77d3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:58.535762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 15/27] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:41:58.605888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d11c5d-dddc-444b-a7e0-a558a47f77d3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:58.645240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:58.645874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.646155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.646423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.657746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:41:58.664937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:41:58.665751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:41:58.666498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:41:58.669767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 13/28] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:41:58.675181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7477b7-a16b-4d60-bece-682e122b1209 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:58.769997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:58.798779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7477b7-a16b-4d60-bece-682e122b1209 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:58.842304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:58.842921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:58.843201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:58.843478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:58.844148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:41:58.848532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:58.849956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:58.859471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 16/29] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:41:58.864916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:58.876482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:58.935990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:59.061768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76f43-fe77-4c7d-b32a-92cff17f9a01 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:59.099812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fafff-be19-4915-9b47-4fcc12520889 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:59.102939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:59.103577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.103902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.104180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.112747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:59.128373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:59.129867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:41:59.159783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:41:59.160713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:41:59.161480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:41:59.161480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:41:59.161480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:41:59.162959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 17/30] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:41:59.166393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:59.176374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:59.177010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.177283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.177547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] PATH_INFO: `/v3/users/a8eb78c86bca426c9163deb9994b5cec` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.180666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] RBAC: Authorizing `identity:get_user(user_id=a8eb78c86bca426c9163deb9994b5cec)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:59.182121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:59.183686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 14/31] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:41:58 2022] GET /identity/v3/users/a8eb78c86bca426c9163deb9994b5cec => generated 247 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:41:59.184539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:59.290463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f4a84-dbcc-4a95-bac0-6041ece53b55 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:59.296047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:59.318408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c0316-8c82-4f1b-9596-64a063fedb1e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:59.330899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:59.331786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.332233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.332783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.333606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:41:59.334073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:59.335692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:59.352392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 15/32] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:41:59.353477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:59.361082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:59.361758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.362040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.362315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.363049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:59.364392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:59.464287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9b177-69f0-4b9a-89a5-cfd782ba1c0e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:59.500433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a3235-26e7-4548-9783-79a7aaaa98df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:59.501389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a3235-26e7-4548-9783-79a7aaaa98df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.501834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a3235-26e7-4548-9783-79a7aaaa98df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.502272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a3235-26e7-4548-9783-79a7aaaa98df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.503698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a3235-26e7-4548-9783-79a7aaaa98df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:59.505370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a3235-26e7-4548-9783-79a7aaaa98df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:59.507163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 16/33] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:41:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:41:59.509120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc70e9-53e1-4e05-b745-0f0feca1eaa2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.509629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc70e9-53e1-4e05-b745-0f0feca1eaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.510069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc70e9-53e1-4e05-b745-0f0feca1eaa2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.511509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 17/34] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:41:59.512453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc70e9-53e1-4e05-b745-0f0feca1eaa2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:59.535037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 18/35] 10.222.0.50 () {60 vars in 1131 bytes} [Tue Aug 30 01:41:59 2022] POST /identity/v3/roles => generated 219 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:41:59.536829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:41:59.555961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:59.614494 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:41:59.659951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc70e9-53e1-4e05-b745-0f0feca1eaa2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:59.703434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7433e36-3424-4caf-a0ea-0d842fa2b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:59.704357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7433e36-3424-4caf-a0ea-0d842fa2b308 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.705447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7433e36-3424-4caf-a0ea-0d842fa2b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.705923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7433e36-3424-4caf-a0ea-0d842fa2b308 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.706973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7433e36-3424-4caf-a0ea-0d842fa2b308 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:59.708777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7433e36-3424-4caf-a0ea-0d842fa2b308 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:59.745726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c397ef-59d5-42e6-8942-f71dcef5b1ca None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:41:59.808152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff61ee-b648-4a2a-be3d-a19619e75004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:41:59.809095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff61ee-b648-4a2a-be3d-a19619e75004 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.809473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff61ee-b648-4a2a-be3d-a19619e75004 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.809829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff61ee-b648-4a2a-be3d-a19619e75004 None admin] PATH_INFO: `/v3/domains/default/users/a8eb78c86bca426c9163deb9994b5cec/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.812956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff61ee-b648-4a2a-be3d-a19619e75004 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a8eb78c86bca426c9163deb9994b5cec, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:41:59.814593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff61ee-b648-4a2a-be3d-a19619e75004 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:41:59.834816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 19/36] 10.222.0.50 () {58 vars in 1361 bytes} [Tue Aug 30 01:41:59 2022] PUT /identity/v3/domains/default/users/a8eb78c86bca426c9163deb9994b5cec/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:41:59.836775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f744a-0061-4f15-93b0-f145970425b7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.837131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f744a-0061-4f15-93b0-f145970425b7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.837476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f744a-0061-4f15-93b0-f145970425b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.859653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-385f744a-0061-4f15-93b0-f145970425b7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:41:59.861768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-385f744a-0061-4f15-93b0-f145970425b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:41:59.886199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 18/37] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:41:59 2022] POST /identity/v3/projects => generated 301 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:41:59.964062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 20/38] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:41:59.964804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2a2f4-f173-4f22-bf58-353b85ae8d86 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.965339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2a2f4-f173-4f22-bf58-353b85ae8d86 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.965908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2a2f4-f173-4f22-bf58-353b85ae8d86 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.967869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 19/39] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:41:59.973380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168034e-ebb3-4147-8378-0fd8819e735d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.973660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168034e-ebb3-4147-8378-0fd8819e735d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.974080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168034e-ebb3-4147-8378-0fd8819e735d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.992102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ce011-c75d-4be7-978a-bf0527ab88de None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:41:59.992425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ce011-c75d-4be7-978a-bf0527ab88de None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:41:59.993259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ce011-c75d-4be7-978a-bf0527ab88de None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:41:59.994869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 20/40] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:00.000660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe9e0e-89f3-414f-b0b7-394944e83088 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:00.000957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe9e0e-89f3-414f-b0b7-394944e83088 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:00.001773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe9e0e-89f3-414f-b0b7-394944e83088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:00.018062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4168034e-ebb3-4147-8378-0fd8819e735d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:00.018664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4168034e-ebb3-4147-8378-0fd8819e735d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:00.025326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-07fe9e0e-89f3-414f-b0b7-394944e83088 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:00.027150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-07fe9e0e-89f3-414f-b0b7-394944e83088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:00.074370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:00.131204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 21/41] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:00.138316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 21/42] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:00.287699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168034e-ebb3-4147-8378-0fd8819e735d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:00.392654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168034e-ebb3-4147-8378-0fd8819e735d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:00.425842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe9e0e-89f3-414f-b0b7-394944e83088 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:00.429218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:00.431502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:00.431936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:00.432398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:00.433231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:00.434860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:00.443401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:00.458682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 22/43] 10.222.0.50 () {60 vars in 1131 bytes} [Tue Aug 30 01:42:00 2022] POST /identity/v3/roles => generated 225 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:00.459506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:00.500895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:00.569772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe9e0e-89f3-414f-b0b7-394944e83088 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:00.617026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:00.618382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:00.618664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:00.618936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:00.620165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:00.621827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:00.623673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 22/44] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:00.629231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:00.708904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240646c2-66ad-4de0-b571-19bb39ad77a9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:00.723947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:00.744361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a2306-dac3-49c5-8bad-f0eb20169559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:00.744976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a2306-dac3-49c5-8bad-f0eb20169559 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:00.745074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7e1b6-f1bb-4a2a-a1c3-70be05452445 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:00.745250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a2306-dac3-49c5-8bad-f0eb20169559 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:00.745555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a2306-dac3-49c5-8bad-f0eb20169559 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:00.746219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a2306-dac3-49c5-8bad-f0eb20169559 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:00.747614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a2306-dac3-49c5-8bad-f0eb20169559 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:00.762004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 23/45] 10.222.0.50 () {60 vars in 1131 bytes} [Tue Aug 30 01:42:00 2022] POST /identity/v3/roles => generated 223 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:00.781462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2f6ce-fd1d-4186-97e1-9961e488c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:00.782109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2f6ce-fd1d-4186-97e1-9961e488c63c None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:00.782391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2f6ce-fd1d-4186-97e1-9961e488c63c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:00.782722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2f6ce-fd1d-4186-97e1-9961e488c63c None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:00.783392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2f6ce-fd1d-4186-97e1-9961e488c63c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:00.784962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2f6ce-fd1d-4186-97e1-9961e488c63c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:00.820277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 23/46] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:42:00 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:42:01.895312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b837d33-86e4-4724-a84e-38ce73089ee8 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:01.895832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b837d33-86e4-4724-a84e-38ce73089ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:01.895832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b837d33-86e4-4724-a84e-38ce73089ee8 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:01.897294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 24/47] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:01.902690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c135-8f9e-4fec-aaa9-0a9d159d89ef None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:01.903126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c135-8f9e-4fec-aaa9-0a9d159d89ef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:01.903254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c135-8f9e-4fec-aaa9-0a9d159d89ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:01.922653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2ad1c135-8f9e-4fec-aaa9-0a9d159d89ef None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:01.923864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2ad1c135-8f9e-4fec-aaa9-0a9d159d89ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:01.990364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 24/48] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:02.256341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b837d33-86e4-4724-a84e-38ce73089ee8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:02.349470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:02.363736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b837d33-86e4-4724-a84e-38ce73089ee8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:02.397164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:02.397796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:02.398078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:02.398355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:02.408915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:02.413876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:02.414655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:02.415404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 25/49] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:02.422028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c135-8f9e-4fec-aaa9-0a9d159d89ef None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:02.482114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:02.521242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c135-8f9e-4fec-aaa9-0a9d159d89ef None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:02.554037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:02.554643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:02.554949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:02.555227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:02.555892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:02.556198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:02.557622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:02.569136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 25/50] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:02.575610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:02.674270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea75a26-c37e-4681-97c7-5193e73078ad None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:02.708036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcf014-1f64-4580-b140-461baea1c3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:02.708687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcf014-1f64-4580-b140-461baea1c3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:02.708974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcf014-1f64-4580-b140-461baea1c3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:02.709252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcf014-1f64-4580-b140-461baea1c3f8 None admin] PATH_INFO: `/v3/users/a8eb78c86bca426c9163deb9994b5cec` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:02.710379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcf014-1f64-4580-b140-461baea1c3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=a8eb78c86bca426c9163deb9994b5cec)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:02.711872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcf014-1f64-4580-b140-461baea1c3f8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:02.713650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 26/51] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:02 2022] GET /identity/v3/users/a8eb78c86bca426c9163deb9994b5cec => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:02.718982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:02.744397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c023d0-8046-4d71-9baa-a303d63cdfd8 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:02.744682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c023d0-8046-4d71-9baa-a303d63cdfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:02.744959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c023d0-8046-4d71-9baa-a303d63cdfd8 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:02.746296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 27/52] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:02.751587 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef26f7-bd81-4f20-bcd8-dc34f745f295 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:02.751873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef26f7-bd81-4f20-bcd8-dc34f745f295 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:02.752326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef26f7-bd81-4f20-bcd8-dc34f745f295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:02.770161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-84ef26f7-bd81-4f20-bcd8-dc34f745f295 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:02.771376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-84ef26f7-bd81-4f20-bcd8-dc34f745f295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:02.782183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:02.821705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e21d0-1234-4821-8751-cedc25afa17b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:02.837727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 28/53] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:02.847386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:02.856368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:02.856999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:02.857284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:02.857560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:02.858666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:02.860253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:02.862066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 26/54] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:02.868208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef26f7-bd81-4f20-bcd8-dc34f745f295 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:02.964255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef26f7-bd81-4f20-bcd8-dc34f745f295 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:02.997210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:02.997815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:02.998095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:02.998368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:02.999092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.domain.id=default, user.id=a8eb78c86bca426c9163deb9994b5cec) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:02.999401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:03.000903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:03.006496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 29/55] 10.222.0.50 () {56 vars in 1322 bytes} [Tue Aug 30 01:42:02 2022] GET /identity/v3/role_assignments?user.id=a8eb78c86bca426c9163deb9994b5cec&scope.domain.id=default&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 530 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:03.108366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:03.311389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f95b3-4829-41be-9a3b-95fc1051b1ae None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:03.342540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b928e0-d890-4892-b47d-37e332adc606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:03.343159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b928e0-d890-4892-b47d-37e332adc606 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:03.343436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b928e0-d890-4892-b47d-37e332adc606 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:03.343704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b928e0-d890-4892-b47d-37e332adc606 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:03.344804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b928e0-d890-4892-b47d-37e332adc606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:03.346231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b928e0-d890-4892-b47d-37e332adc606 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:03.347764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 27/56] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:03.353847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:03.367040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:03.450298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881161e4-df01-439e-8f9c-26f164983a36 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:03.482364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1d1c-a956-4c6b-88bb-fd998060b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:03.482958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1d1c-a956-4c6b-88bb-fd998060b936 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:03.483235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1d1c-a956-4c6b-88bb-fd998060b936 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:03.483506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1d1c-a956-4c6b-88bb-fd998060b936 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:03.484151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1d1c-a956-4c6b-88bb-fd998060b936 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:03.485871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1d1c-a956-4c6b-88bb-fd998060b936 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:03.512166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 30/57] 10.222.0.50 () {60 vars in 1140 bytes} [Tue Aug 30 01:42:03 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:05.371968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c37297-0124-4a54-aaa2-81503007dea9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:05.372490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c37297-0124-4a54-aaa2-81503007dea9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:05.372575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c37297-0124-4a54-aaa2-81503007dea9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:05.373950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 28/58] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:05.379009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d3f5-338b-4ba1-aeb9-79bd4e0b7d5e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:05.379398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d3f5-338b-4ba1-aeb9-79bd4e0b7d5e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:05.379559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d3f5-338b-4ba1-aeb9-79bd4e0b7d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:05.397596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0d41d3f5-338b-4ba1-aeb9-79bd4e0b7d5e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:05.399314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0d41d3f5-338b-4ba1-aeb9-79bd4e0b7d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:05.452179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:05.463572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 31/59] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:05.726540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c37297-0124-4a54-aaa2-81503007dea9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:05.787327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:05.827753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c37297-0124-4a54-aaa2-81503007dea9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:05.859167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdc0f2-2a4e-44cb-a446-5d10d7cf93fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:05.859793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdc0f2-2a4e-44cb-a446-5d10d7cf93fa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:05.860078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdc0f2-2a4e-44cb-a446-5d10d7cf93fa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:05.860391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdc0f2-2a4e-44cb-a446-5d10d7cf93fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:05.861774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdc0f2-2a4e-44cb-a446-5d10d7cf93fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:05.863216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdc0f2-2a4e-44cb-a446-5d10d7cf93fa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:05.864801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 29/60] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:05.870811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d3f5-338b-4ba1-aeb9-79bd4e0b7d5e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:05.963925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d3f5-338b-4ba1-aeb9-79bd4e0b7d5e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:05.996166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f981d42-0ee6-462a-ab8c-381df4271b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:05.996904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f981d42-0ee6-462a-ab8c-381df4271b56 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:05.997183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f981d42-0ee6-462a-ab8c-381df4271b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:05.997454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f981d42-0ee6-462a-ab8c-381df4271b56 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:05.998094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f981d42-0ee6-462a-ab8c-381df4271b56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:05.999533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f981d42-0ee6-462a-ab8c-381df4271b56 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:06.025793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 32/61] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:42:05 2022] POST /identity/v3/users => generated 275 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:08.939322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4216-48bd-4a49-9810-701940185ab9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:08.939834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4216-48bd-4a49-9810-701940185ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:08.939906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4216-48bd-4a49-9810-701940185ab9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:08.941247 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 30/62] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:08.948304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b308e-6cf7-4931-b191-7cb5a7e74ddb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:08.948731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b308e-6cf7-4931-b191-7cb5a7e74ddb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:08.948902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b308e-6cf7-4931-b191-7cb5a7e74ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:08.977284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-516b308e-6cf7-4931-b191-7cb5a7e74ddb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:08.978471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-516b308e-6cf7-4931-b191-7cb5a7e74ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:09.041763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:09.077581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 33/63] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:09.165264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac524a-c43a-4f62-8c4f-a9cb8c5deb8e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.165885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac524a-c43a-4f62-8c4f-a9cb8c5deb8e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.166374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac524a-c43a-4f62-8c4f-a9cb8c5deb8e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.167907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 31/64] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:09.173306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6e758-073c-4c26-9fe2-27879b593a05 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.173840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6e758-073c-4c26-9fe2-27879b593a05 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.174311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6e758-073c-4c26-9fe2-27879b593a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.200588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8db6e758-073c-4c26-9fe2-27879b593a05 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:09.202066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8db6e758-073c-4c26-9fe2-27879b593a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:09.311869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 34/65] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:09.391783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac524a-c43a-4f62-8c4f-a9cb8c5deb8e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:09.502027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027913-456e-4776-84e5-98f8a52495ab None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.502645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027913-456e-4776-84e5-98f8a52495ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.503114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027913-456e-4776-84e5-98f8a52495ab None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.504950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 35/66] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:09.509588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cebf0-5273-40c8-80ab-18d1dc8094d7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.509876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cebf0-5273-40c8-80ab-18d1dc8094d7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.510155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cebf0-5273-40c8-80ab-18d1dc8094d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.538804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3f3cebf0-5273-40c8-80ab-18d1dc8094d7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:09.540585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3f3cebf0-5273-40c8-80ab-18d1dc8094d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:09.544092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac524a-c43a-4f62-8c4f-a9cb8c5deb8e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:09.585968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:09.593097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0b001-710d-43e8-9a3b-0931415a38d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:09.593736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0b001-710d-43e8-9a3b-0931415a38d1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.594017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0b001-710d-43e8-9a3b-0931415a38d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.594291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0b001-710d-43e8-9a3b-0931415a38d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.609926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0b001-710d-43e8-9a3b-0931415a38d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-09f0b001-710d-43e8-9a3b-0931415a38d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:09.615238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:09.616108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:09.616865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:09.619097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 32/67] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:09.620474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c14346-b174-4e5c-80a9-83a358cfd588 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.620966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c14346-b174-4e5c-80a9-83a358cfd588 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.621437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c14346-b174-4e5c-80a9-83a358cfd588 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.622850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 33/68] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:09.625413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c14346-b174-4e5c-80a9-83a358cfd588 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:09.638294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 36/69] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:09.639620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1495ed-2edf-4733-8e3d-80a057f17c47 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.640465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1495ed-2edf-4733-8e3d-80a057f17c47 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.640887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1495ed-2edf-4733-8e3d-80a057f17c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.664565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-be1495ed-2edf-4733-8e3d-80a057f17c47 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:09.666653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be1495ed-2edf-4733-8e3d-80a057f17c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:09.701835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:09.735868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c14346-b174-4e5c-80a9-83a358cfd588 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:09.741786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 37/70] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:09.742805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1495ed-2edf-4733-8e3d-80a057f17c47 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:09.781626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f9ddf-737c-4c8d-bad0-812cb732b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:09.782440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9ddf-737c-4c8d-bad0-812cb732b67c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.782879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9ddf-737c-4c8d-bad0-812cb732b67c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.783291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f9ddf-737c-4c8d-bad0-812cb732b67c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.784078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9ddf-737c-4c8d-bad0-812cb732b67c None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:09.784542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9ddf-737c-4c8d-bad0-812cb732b67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:09.786048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f9ddf-737c-4c8d-bad0-812cb732b67c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:09.795343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 34/71] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:09.796689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad52d9-0f74-4641-91c2-863647aba73a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.797112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad52d9-0f74-4641-91c2-863647aba73a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.797544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad52d9-0f74-4641-91c2-863647aba73a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.798886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 35/72] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:09.801237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad52d9-0f74-4641-91c2-863647aba73a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:09.858726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:09.863376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1495ed-2edf-4733-8e3d-80a057f17c47 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:09.906788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b3459-ae85-48c5-85aa-aaaabfa725f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:09.907433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b3459-ae85-48c5-85aa-aaaabfa725f6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.907713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b3459-ae85-48c5-85aa-aaaabfa725f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.908260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b3459-ae85-48c5-85aa-aaaabfa725f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.911083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad52d9-0f74-4641-91c2-863647aba73a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:09.920519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b3459-ae85-48c5-85aa-aaaabfa725f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-024b3459-ae85-48c5-85aa-aaaabfa725f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:09.927135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:09.927977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:09.928865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 38/73] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:09.930430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad071dc-4302-47cb-90a0-cbf1e7c0f321 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.930842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad071dc-4302-47cb-90a0-cbf1e7c0f321 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.931293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad071dc-4302-47cb-90a0-cbf1e7c0f321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.946597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:09.947227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:09.947558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:09.947835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:09.952860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dad071dc-4302-47cb-90a0-cbf1e7c0f321 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:09.954097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dad071dc-4302-47cb-90a0-cbf1e7c0f321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:09.966205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:09.967668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:09.969359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 36/74] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:09 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:09.970135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:09.985148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.030981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 39/75] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:10.032186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad071dc-4302-47cb-90a0-cbf1e7c0f321 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.091525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cf125-157f-4934-86ac-0903371dba1e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.107401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.134518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad071dc-4302-47cb-90a0-cbf1e7c0f321 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.137528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.138165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.138446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.138723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.151008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:10.155852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:10.156849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 37/76] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:10.157788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.174693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.175499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.176045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.176880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.177945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:10.178413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.180927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.189710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 40/77] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:10.190294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.211569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.271322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed092c12-b894-440f-b6ed-678dafe6da42 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.293579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482bef5-7461-4419-82e9-a917c17afe4e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.311485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979d986-8653-4c80-9768-36233d889fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.312099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979d986-8653-4c80-9768-36233d889fee None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.312417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979d986-8653-4c80-9768-36233d889fee None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.312699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979d986-8653-4c80-9768-36233d889fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.329495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.330124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.330403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.330678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.340774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:10.345669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:10.346674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:10.347466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 41/78] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:10.349039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.353199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.382358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979d986-8653-4c80-9768-36233d889fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.384425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979d986-8653-4c80-9768-36233d889fee None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.441012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.460724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d73d-e78a-44d6-a918-f33f5dd1ff6b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.483390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 38/79] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:10.484232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979d986-8653-4c80-9768-36233d889fee None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.494256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.494864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.495146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.495423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.496092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:10.496440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.497848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.505692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 42/80] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:10.506467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.559306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.589526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979d986-8653-4c80-9768-36233d889fee None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.602668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88fb49-92c6-4a90-8961-553bcc2493e1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.624156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.624816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.625100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.625380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] PATH_INFO: `/v3/users/a8eb78c86bca426c9163deb9994b5cec` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.626503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] RBAC: Authorizing `identity:get_user(user_id=a8eb78c86bca426c9163deb9994b5cec)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.627971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.629555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 39/81] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/users/a8eb78c86bca426c9163deb9994b5cec => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:10.631406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.636196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.636851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.637131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.637406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.638510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.639938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.641529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 43/82] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:10.642285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.654322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.732897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe57c2-5d1a-47f7-8b44-f34053637511 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.739231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49b116-a897-4bd1-95a2-14c88d66e3c8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.767114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.767722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.768005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.768306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.768969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:10.769269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.770647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.772080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.772742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.773024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.773301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.774443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.775480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.775853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.782320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 44/83] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:10.782320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.782535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 40/84] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:10.782535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.856227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.877598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba94dd-6172-4c02-b804-0204e9d0652e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.878568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cfc68-938f-4b0c-845e-4af94fadadad None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:10.910984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.911613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.911894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.912171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.912379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:10.912987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:10.913271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:10.913361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.913550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:10.914802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:10.916401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 45/85] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:10.916401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:10.981882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:10.984162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:10.986036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.013395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f1713-e6cc-49bc-b960-84b101e35170 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:11.046913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:11.047513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:11.047796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:11.048076 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:11.048763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:11.050214 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.070666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 46/86] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:42:10 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:11.073180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:11.075996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:11.084050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 41/87] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:11.085180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:11.189044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd3379-3ea0-47f2-81ea-f3b7b72102d4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:11.206741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:11.210039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762729f-bc53-49ac-8f57-ffd677aac331 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:11.229001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:11.229628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:11.229910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:11.230186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:11.230941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=801bea5ab5ed40fd9053f989885b4314) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:11.231225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:11.233545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.241178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 47/88] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/role_assignments?user.id=801bea5ab5ed40fd9053f989885b4314&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=61a194f6aba541de8c32f53dfc572624 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:11.242075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:11.255620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:11.256442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:11.256713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:11.256985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:11.258742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:11.260489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.263390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 42/89] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:11.264275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:11.334300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:11.390344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786fbba-4836-4e3d-a138-076fa8c00b7e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:11.420686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc5ea1-19fc-4dc6-8876-30fee51de01a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:11.433698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:11.434419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:11.434701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:11.435032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:11.465871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:11.466447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:11.466727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:11.467002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:11.468170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:11.469646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.471463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 43/90] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:11.472188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:11.518497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:11.520447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:11.522272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.595501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706f8c-e925-49a0-9281-5b637212bb57 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:11.620208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:11.641596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 48/91] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:11.649323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:11.695803 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:11.700949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:11.701453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:11.701947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:11.800607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:11.802319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.838548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:11.842432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a3698-cabc-42a5-aefa-30b21e4a1207 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:11.909798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 44/92] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:11.910755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:11.932985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:11.933617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:11.933903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:11.934184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:11.934926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=a8eb78c86bca426c9163deb9994b5cec) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:11.935238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:11.936781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:11.954356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 49/93] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/role_assignments?user.id=a8eb78c86bca426c9163deb9994b5cec&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:11.955103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:12.026925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:12.056140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071962a8-67df-4281-974b-7b43170ec527 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:12.127824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:12.128521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:12.128800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:12.129073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:12.130909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:12.132386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:12.136428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 45/94] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:12.144382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:12.148659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:12.180400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153130a-867d-47c2-932a-6c4fc5e40feb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:12.249732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:12.250373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:12.250654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:12.250928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] PATH_INFO: `/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:12.252998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] RBAC: Authorizing `identity:get_project(project_id=3ddac4aad8804629b3ec37c0bdd42cb8)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:12.254701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:12.256814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 50/95] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:11 2022] GET /identity/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8 => generated 312 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:12.263178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:12.289189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d6b1-d31f-4b7b-96bb-71cffc5cf608 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:12.323627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254982f-8992-4675-b074-665aadf55e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:12.324418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254982f-8992-4675-b074-665aadf55e01 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:12.324893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254982f-8992-4675-b074-665aadf55e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:12.325017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254982f-8992-4675-b074-665aadf55e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:12.325696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254982f-8992-4675-b074-665aadf55e01 None admin] RBAC: Adding query filter params (role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=801bea5ab5ed40fd9053f989885b4314) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:12.326016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254982f-8992-4675-b074-665aadf55e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:12.327485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254982f-8992-4675-b074-665aadf55e01 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:12.340268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 46/96] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:12 2022] GET /identity/v3/role_assignments?user.id=801bea5ab5ed40fd9053f989885b4314&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=fdf7d3a219d340db99038e400383416b => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:12.432912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:12.438167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e865d-1189-4923-8d34-b65108e1422d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:12.477926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd78d87-b53f-4920-b317-9058e995a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:12.478926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78d87-b53f-4920-b317-9058e995a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:12.479403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78d87-b53f-4920-b317-9058e995a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:12.479901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78d87-b53f-4920-b317-9058e995a5e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:12.480894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd78d87-b53f-4920-b317-9058e995a5e9 None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=3ddac4aad8804629b3ec37c0bdd42cb8, user.id=801bea5ab5ed40fd9053f989885b4314) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:12.481416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd78d87-b53f-4920-b317-9058e995a5e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:12.483105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd78d87-b53f-4920-b317-9058e995a5e9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:12.489881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 51/97] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:12 2022] GET /identity/v3/role_assignments?user.id=801bea5ab5ed40fd9053f989885b4314&scope.project.id=3ddac4aad8804629b3ec37c0bdd42cb8&role.id=61a194f6aba541de8c32f53dfc572624 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:13.706924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ddf36-7b16-4a88-9768-26ea198034f5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:13.707711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ddf36-7b16-4a88-9768-26ea198034f5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:13.708111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ddf36-7b16-4a88-9768-26ea198034f5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:13.709880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 47/98] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:13.715823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38394d55-caaa-456c-8c53-5012f77012a5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:13.716294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38394d55-caaa-456c-8c53-5012f77012a5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:13.716411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38394d55-caaa-456c-8c53-5012f77012a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:13.743960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-38394d55-caaa-456c-8c53-5012f77012a5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:13.745218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-38394d55-caaa-456c-8c53-5012f77012a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:13.810630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1218c7-c014-48ea-91fd-dc8139257709 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:13.811124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1218c7-c014-48ea-91fd-dc8139257709 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:13.811582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1218c7-c014-48ea-91fd-dc8139257709 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:13.813184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 48/99] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:13.822417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb26e-6df8-45ae-83c2-71c1beb04486 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:13.822901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb26e-6df8-45ae-83c2-71c1beb04486 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:13.823315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cb26e-6df8-45ae-83c2-71c1beb04486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:13.844498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 52/100] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:13.858478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff157a51-e31d-4abf-8ab1-4060a655cb7b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:13.858783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff157a51-e31d-4abf-8ab1-4060a655cb7b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:13.859045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff157a51-e31d-4abf-8ab1-4060a655cb7b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:13.860836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 53/101] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:13.865077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4e7cb26e-6df8-45ae-83c2-71c1beb04486 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:13.866284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e7cb26e-6df8-45ae-83c2-71c1beb04486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:13.869859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508bc9f-f057-400c-90f6-c672476bd4cd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:13.870142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508bc9f-f057-400c-90f6-c672476bd4cd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:13.870415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508bc9f-f057-400c-90f6-c672476bd4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:13.907229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3508bc9f-f057-400c-90f6-c672476bd4cd None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:13.908446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3508bc9f-f057-400c-90f6-c672476bd4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:14.029415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:14.039759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 49/102] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:14.073750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 54/103] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:14.192052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cb26e-6df8-45ae-83c2-71c1beb04486 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:14.321788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cb26e-6df8-45ae-83c2-71c1beb04486 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:14.375308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:14.375950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:14.376229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:14.376529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:14.377717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:14.379173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:14.380967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 50/104] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:14.386464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508bc9f-f057-400c-90f6-c672476bd4cd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:14.415790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:14.450413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:14.501115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508bc9f-f057-400c-90f6-c672476bd4cd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:14.538386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:14.539015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:14.544456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:14.544643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:14.544905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:14.545880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:14.547840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:14.583964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2f4e8-8020-4d62-a5bc-249f646461f5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:14.616297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 55/105] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:42:14 2022] POST /identity/v3/projects => generated 302 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:14.617320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:14.632989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:14.634032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:14.634504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:14.634966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:14.664869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:14.670195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:14.671039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:14.671769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:14.674288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 51/106] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:14.692843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:14.744633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:14.798240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2feb71-1d47-4253-b13c-fcd7a7c49bc5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:14.819323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98199fa0-edf0-4224-840f-710d8443b77a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:14.866942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:14.879094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:14.879741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:14.880017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:14.880308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:14.893164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:14.893912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:14.894198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:14.894475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:14.895162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:14.895467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:14.896896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:14.916324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 52/107] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:14.921246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:14.926880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:14.927691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:14.928472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 56/108] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:14.930069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:14.937218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:15.091702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3782f1e-6e08-4255-8581-e4e35e54bd10 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:15.096752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948be15-4956-4155-99a6-7a1fa59a5b89 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:15.129645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c4182-b2da-4121-8b48-fd6804e9dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:15.130282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4182-b2da-4121-8b48-fd6804e9dd3a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.130559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4182-b2da-4121-8b48-fd6804e9dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.130838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c4182-b2da-4121-8b48-fd6804e9dd3a None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.132020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c4182-b2da-4121-8b48-fd6804e9dd3a None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:15.134087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c4182-b2da-4121-8b48-fd6804e9dd3a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:15.134567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c07fe-ffc9-42e9-90a8-82a782b9394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:15.135407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c07fe-ffc9-42e9-90a8-82a782b9394a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.135680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 53/109] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:14 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:15.136088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c07fe-ffc9-42e9-90a8-82a782b9394a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.136598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c07fe-ffc9-42e9-90a8-82a782b9394a None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.137519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c07fe-ffc9-42e9-90a8-82a782b9394a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:15.138032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c07fe-ffc9-42e9-90a8-82a782b9394a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:15.139609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c07fe-ffc9-42e9-90a8-82a782b9394a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:15.141367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:15.142786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a8b54-a042-4440-8c98-fad00b67a7b2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.143108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a8b54-a042-4440-8c98-fad00b67a7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.143407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a8b54-a042-4440-8c98-fad00b67a7b2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.149573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 54/110] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:15.149573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a8b54-a042-4440-8c98-fad00b67a7b2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:15.158346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 57/111] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:15.159881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac044229-aab0-4d0e-95ad-e861d4301ce0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.160519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac044229-aab0-4d0e-95ad-e861d4301ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.161017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac044229-aab0-4d0e-95ad-e861d4301ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.184393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ac044229-aab0-4d0e-95ad-e861d4301ce0 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:15.185911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ac044229-aab0-4d0e-95ad-e861d4301ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:15.212705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:15.259729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a8b54-a042-4440-8c98-fad00b67a7b2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:15.260859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 58/112] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:15.261921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac044229-aab0-4d0e-95ad-e861d4301ce0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:15.295475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b9119-bf45-4d13-a8a7-c6fc6b4989ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:15.296120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9119-bf45-4d13-a8a7-c6fc6b4989ed None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.296428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9119-bf45-4d13-a8a7-c6fc6b4989ed None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.296699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9119-bf45-4d13-a8a7-c6fc6b4989ed None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.297957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b9119-bf45-4d13-a8a7-c6fc6b4989ed None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:15.299566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b9119-bf45-4d13-a8a7-c6fc6b4989ed None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:15.301279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 55/113] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:15 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:15.302556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2c721-ee1d-4ba9-8ee0-983ac752a535 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.302842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2c721-ee1d-4ba9-8ee0-983ac752a535 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.312290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2c721-ee1d-4ba9-8ee0-983ac752a535 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.312290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 56/114] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:15.313440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2c721-ee1d-4ba9-8ee0-983ac752a535 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:15.456570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac044229-aab0-4d0e-95ad-e861d4301ce0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:15.501417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2c721-ee1d-4ba9-8ee0-983ac752a535 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:15.525821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:15.528689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:15.540134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a16b8-86f6-4ffd-b857-1c3920fffa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:15.541212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a16b8-86f6-4ffd-b857-1c3920fffa1c None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.541631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a16b8-86f6-4ffd-b857-1c3920fffa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.542096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a16b8-86f6-4ffd-b857-1c3920fffa1c None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/801bea5ab5ed40fd9053f989885b4314/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.544196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a16b8-86f6-4ffd-b857-1c3920fffa1c None admin] RBAC: Authorizing `identity:create_grant(project_id=8b71340f7ded406b9f92818f9e00a227, user_id=801bea5ab5ed40fd9053f989885b4314, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:15.546015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a16b8-86f6-4ffd-b857-1c3920fffa1c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:15.556898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398f4f8-872b-4570-b49b-79fb51800d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:15.557592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398f4f8-872b-4570-b49b-79fb51800d5d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.557881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398f4f8-872b-4570-b49b-79fb51800d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.558159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398f4f8-872b-4570-b49b-79fb51800d5d None admin] PATH_INFO: `/v3/users/7dab5b95f6784a02baf0bd1a2e4a2de6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.572757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 57/115] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:15 2022] PUT /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/801bea5ab5ed40fd9053f989885b4314/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:15.574133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b16754-2aed-49b3-a853-fb0e32dfa0c6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.574423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b16754-2aed-49b3-a853-fb0e32dfa0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.574708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b16754-2aed-49b3-a853-fb0e32dfa0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.587467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398f4f8-872b-4570-b49b-79fb51800d5d None admin] RBAC: Authorizing `identity:get_user(user_id=7dab5b95f6784a02baf0bd1a2e4a2de6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:15.589007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398f4f8-872b-4570-b49b-79fb51800d5d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:15.590770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 59/116] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:15 2022] GET /identity/v3/users/7dab5b95f6784a02baf0bd1a2e4a2de6 => generated 289 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:15.598611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783bfa3-02bf-4af1-bdbc-65380f2aeb03 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.598951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783bfa3-02bf-4af1-bdbc-65380f2aeb03 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.599231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783bfa3-02bf-4af1-bdbc-65380f2aeb03 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.606903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 60/117] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:15.607921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783bfa3-02bf-4af1-bdbc-65380f2aeb03 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:15.615504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-26b16754-2aed-49b3-a853-fb0e32dfa0c6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:15.617376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-26b16754-2aed-49b3-a853-fb0e32dfa0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:15.721012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 58/118] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:15.721872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b16754-2aed-49b3-a853-fb0e32dfa0c6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:15.734428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783bfa3-02bf-4af1-bdbc-65380f2aeb03 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:15.774106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:15.774853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.775057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.775335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.834184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:15.841835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b16754-2aed-49b3-a853-fb0e32dfa0c6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:15.851931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:15.853449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:15.864635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:15.879494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58fb3d-aaf9-43e1-a67a-33f1b47788c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:15.880506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fb3d-aaf9-43e1-a67a-33f1b47788c8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.881055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fb3d-aaf9-43e1-a67a-33f1b47788c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.881670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fb3d-aaf9-43e1-a67a-33f1b47788c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.893461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58fb3d-aaf9-43e1-a67a-33f1b47788c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-aa58fb3d-aaf9-43e1-a67a-33f1b47788c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:15.899501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:15.900353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:15.901103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:15.903435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 59/119] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:15.905531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519ab2d-99f6-483a-8efd-918a61fe44f3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:15.906109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519ab2d-99f6-483a-8efd-918a61fe44f3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:15.906581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519ab2d-99f6-483a-8efd-918a61fe44f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:15.925077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e519ab2d-99f6-483a-8efd-918a61fe44f3 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:15.926650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e519ab2d-99f6-483a-8efd-918a61fe44f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:16.000954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 61/120] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:16.005358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.035535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 60/121] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:16.036449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519ab2d-99f6-483a-8efd-918a61fe44f3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.185176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:16.222388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcae46-8f23-4cb6-8bad-efa5da4adc2f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.235041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519ab2d-99f6-483a-8efd-918a61fe44f3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.237826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:16.299246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.301749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.302185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.302758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.304410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:16.305965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:16.309239 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:16.314308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.318370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.318809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.319242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.321029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:16.322718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:16.323192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 62/122] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:16.324116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.325433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 61/123] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:16.326535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.448555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244806f-bbf6-4755-b07b-fff633d88af0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.460341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab10b6-1df1-4fd7-94c8-c798f38aaff6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.487108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.488085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.488599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.489065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.499578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.500186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:16.500317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.500559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.500844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:16.506357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:16.507271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:16.508022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:16.509924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 62/124] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:16.511908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.515773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:16.517810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:16.518651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:16.519622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 63/125] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:16.520451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.529455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:16.552384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:16.623074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c82ac-ad66-4b72-8e5e-796f3515f527 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.634730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060720b-ac8d-4378-9362-de4eb73d6116 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.657233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.658185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.658657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.659142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] PATH_INFO: `/v3/users/a8eb78c86bca426c9163deb9994b5cec` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.660786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] RBAC: Authorizing `identity:get_user(user_id=a8eb78c86bca426c9163deb9994b5cec)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:16.662419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:16.665064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 63/126] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/users/a8eb78c86bca426c9163deb9994b5cec => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:16.666018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.675817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.676469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.676755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.677034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.677773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] RBAC: Adding query filter params (role.id=5615ab8f7bfd40e5a904031e70bcc62b, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=7dab5b95f6784a02baf0bd1a2e4a2de6) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:16.678084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:16.679703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:16.686434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 64/127] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/role_assignments?user.id=7dab5b95f6784a02baf0bd1a2e4a2de6&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=5615ab8f7bfd40e5a904031e70bcc62b => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:16.687055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.780856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891cc78-489b-4603-8ef9-209f98f63d2a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.785144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:16.836781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:16.854412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.855106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.855106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.856001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.856001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:16.856381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:16.857759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:16.883654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 64/128] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:16.884634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:16.902010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb04e0-fdb2-4a52-b54d-b075970860f0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:16.985210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:16.986667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:16.987508 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:16.988003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:16.988961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:16.989513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:16.993295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.011506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 65/129] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:17.015165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:17.016603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df1cb-afc9-44ba-9c57-458c74282927 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.058966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.059933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.061514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.061878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.063525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.067186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.068875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 65/130] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:16 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:17.073179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:17.137612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:17.196173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874279e-f799-4768-a049-17281f75ab12 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.221499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1bb71-903f-4079-a4c1-f9f5d2a1da38 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.238925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:17.246967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.247865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.248144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.248684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.250332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.252600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.254592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 66/131] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:17 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:17.256168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:17.257773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870ddab-bc41-4dee-86f0-5486e51c1734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.258966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870ddab-bc41-4dee-86f0-5486e51c1734 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.259247 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870ddab-bc41-4dee-86f0-5486e51c1734 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.259521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870ddab-bc41-4dee-86f0-5486e51c1734 None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.260817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870ddab-bc41-4dee-86f0-5486e51c1734 None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.262313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870ddab-bc41-4dee-86f0-5486e51c1734 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.264380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 66/132] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:17 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:17.265437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bd45-aeea-4bc9-a440-20d6aa5d0a20 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.265729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bd45-aeea-4bc9-a440-20d6aa5d0a20 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.266005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bd45-aeea-4bc9-a440-20d6aa5d0a20 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.267292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 67/133] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:17.267997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bd45-aeea-4bc9-a440-20d6aa5d0a20 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:17.387055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:17.389435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f9808-ea07-40f9-8253-29ee68266bac None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.393471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bd45-aeea-4bc9-a440-20d6aa5d0a20 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.429803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826ce24-3412-410e-a814-6ac02eccc908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.430968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826ce24-3412-410e-a814-6ac02eccc908 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.431080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.431243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826ce24-3412-410e-a814-6ac02eccc908 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.431521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826ce24-3412-410e-a814-6ac02eccc908 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/a8eb78c86bca426c9163deb9994b5cec/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.432169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.432649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.433149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] PATH_INFO: `/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.433562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826ce24-3412-410e-a814-6ac02eccc908 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b71340f7ded406b9f92818f9e00a227, user_id=a8eb78c86bca426c9163deb9994b5cec, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.434669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] RBAC: Authorizing `identity:get_project(project_id=3ddac4aad8804629b3ec37c0bdd42cb8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.435000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826ce24-3412-410e-a814-6ac02eccc908 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.436605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.438720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 68/134] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:17 2022] GET /identity/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:17.439871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:17.451442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 67/135] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:17 2022] PUT /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/a8eb78c86bca426c9163deb9994b5cec/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:17.454073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327c257-79ba-4fa5-a9a7-a9c751163792 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.454627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327c257-79ba-4fa5-a9a7-a9c751163792 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.455074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327c257-79ba-4fa5-a9a7-a9c751163792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.479618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2327c257-79ba-4fa5-a9a7-a9c751163792 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:17.481914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2327c257-79ba-4fa5-a9a7-a9c751163792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:17.590780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:17.608455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 68/136] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:17.609609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327c257-79ba-4fa5-a9a7-a9c751163792 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:17.685072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834fccc-5cc0-41d8-9991-08272efd0c54 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.761377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.762389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.762860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.763321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.764707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.766435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.768295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 69/137] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:17 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:17.775935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:17.815046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:17.817815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327c257-79ba-4fa5-a9a7-a9c751163792 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.884926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2102c9-72ee-46c4-9823-9bde42534a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.886286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2102c9-72ee-46c4-9823-9bde42534a1f None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.886702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2102c9-72ee-46c4-9823-9bde42534a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.887145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2102c9-72ee-46c4-9823-9bde42534a1f None admin] PATH_INFO: `/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8/users/801bea5ab5ed40fd9053f989885b4314/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.889755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a207417-c80b-444c-982a-6df59cbb7b03 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:17.891398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2102c9-72ee-46c4-9823-9bde42534a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=3ddac4aad8804629b3ec37c0bdd42cb8, user_id=801bea5ab5ed40fd9053f989885b4314, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.893023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2102c9-72ee-46c4-9823-9bde42534a1f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.926475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:17.927205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.927336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 69/138] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:17 2022] PUT /identity/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8/users/801bea5ab5ed40fd9053f989885b4314/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:17.927508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.927761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/801bea5ab5ed40fd9053f989885b4314/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.929125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec903c8b-2ffd-4323-8276-b7f0d8bc1c84 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.929546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec903c8b-2ffd-4323-8276-b7f0d8bc1c84 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.929992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec903c8b-2ffd-4323-8276-b7f0d8bc1c84 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.930282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] RBAC: Authorizing `identity:create_grant(project_id=8b71340f7ded406b9f92818f9e00a227, user_id=801bea5ab5ed40fd9053f989885b4314, role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:17.931706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 70/139] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:17.931857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:17.942181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36c066-019a-43bf-ace8-17d888dfa625 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:17.942673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36c066-019a-43bf-ace8-17d888dfa625 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:17.943131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36c066-019a-43bf-ace8-17d888dfa625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:17.953909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 70/140] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:17 2022] PUT /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/801bea5ab5ed40fd9053f989885b4314/roles/fdf7d3a219d340db99038e400383416b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:17.993532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1e36c066-019a-43bf-ace8-17d888dfa625 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:17.995069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1e36c066-019a-43bf-ace8-17d888dfa625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:18.045657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:18.079248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:18.138249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 71/141] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:18.180195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:18.187786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c97a93-65fe-4bb1-9147-ee3ee52d278a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:18.228007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:18.228660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:18.228911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:18.229190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:18.230327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:18.231787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:18.233401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 71/142] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:18.239263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36c066-019a-43bf-ace8-17d888dfa625 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:18.365817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36c066-019a-43bf-ace8-17d888dfa625 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:18.411433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:18.412404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:18.412884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:18.413337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:18.414314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:18.415936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:18.458415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 72/143] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:42:18 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:42:18.624792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:18.747461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bd5a7-8ad3-4aa4-9da5-c4a4c71d62e5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:18.783477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:18.784444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:18.784958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:18.785439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:18.797111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:18.802410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:18.803233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:18.803966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 72/144] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:18.811298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:18.883013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:18.970040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f7df-0b28-420e-96b6-1d5dbfe37a73 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:19.015433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:19.016300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:19.016747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:19.017163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:19.017970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:19.018416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:19.019935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:19.028453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 73/145] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:19.036360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:19.077322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:19.175508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab947286-45c9-4b4d-be65-9b31b8dbf195 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:19.222831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c317a-037e-411b-9ed1-adb124bf3416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:19.223471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c317a-037e-411b-9ed1-adb124bf3416 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:19.223744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c317a-037e-411b-9ed1-adb124bf3416 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:19.224014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c317a-037e-411b-9ed1-adb124bf3416 None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:19.225182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c317a-037e-411b-9ed1-adb124bf3416 None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:19.227259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c317a-037e-411b-9ed1-adb124bf3416 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:19.228886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 73/146] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:19 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:19.235009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:19.259582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a7f2-2568-49fc-ac13-f90e76ebad83 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:19.259912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a7f2-2568-49fc-ac13-f90e76ebad83 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:19.260137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a7f2-2568-49fc-ac13-f90e76ebad83 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:19.261613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 74/147] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:19.267843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a266-d44b-43bc-9304-978ec38466e9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:19.268412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a266-d44b-43bc-9304-978ec38466e9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:19.268993 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a266-d44b-43bc-9304-978ec38466e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:19.293347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-aa03a266-d44b-43bc-9304-978ec38466e9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:19.294894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aa03a266-d44b-43bc-9304-978ec38466e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:19.409942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 75/148] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:19.416711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:19.420900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9450ee-9b26-4db8-9d7c-b284dd95943f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:19.462322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:19.463265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:19.463764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:19.464228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:19.538591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:19.540971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:19.661823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:19.662388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 74/149] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:19.665927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a266-d44b-43bc-9304-978ec38466e9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:19.703277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:19.802817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a266-d44b-43bc-9304-978ec38466e9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:19.838357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:19.848926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:19.849987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:19.850261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:19.850528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:19.852216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:19.853657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:19.855291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 76/150] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:19 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:19.864600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:19.985936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:20.004658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b983b2-d7fb-4704-9819-e273a5635cf5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:20.035839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b806ef9-a080-498b-b8c9-6e92353b73c4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:20.071977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bb780-124b-46e2-b010-80e286597286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:20.072648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bb780-124b-46e2-b010-80e286597286 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.072825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bb780-124b-46e2-b010-80e286597286 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.073092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bb780-124b-46e2-b010-80e286597286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.073830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bb780-124b-46e2-b010-80e286597286 None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=801bea5ab5ed40fd9053f989885b4314) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:20.074137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bb780-124b-46e2-b010-80e286597286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:20.075651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bb780-124b-46e2-b010-80e286597286 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:20.088412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 77/151] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:19 2022] GET /identity/v3/role_assignments?user.id=801bea5ab5ed40fd9053f989885b4314&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=61a194f6aba541de8c32f53dfc572624 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:20.110513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:20.111779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.112286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.112750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.129131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:20.134492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:20.135312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:20.136125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:20.143115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 75/152] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:20.149810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bb780-124b-46e2-b010-80e286597286 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:20.294688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bb780-124b-46e2-b010-80e286597286 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:20.338747 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f88903-dab6-4cd6-95af-def3ddf09796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:20.339282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f88903-dab6-4cd6-95af-def3ddf09796 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.339559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f88903-dab6-4cd6-95af-def3ddf09796 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.339831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f88903-dab6-4cd6-95af-def3ddf09796 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.340538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f88903-dab6-4cd6-95af-def3ddf09796 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:20.340835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f88903-dab6-4cd6-95af-def3ddf09796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:20.342226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f88903-dab6-4cd6-95af-def3ddf09796 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:20.352042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 78/153] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:20.357608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:20.465987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:20.505840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9ea0-fc7f-4a41-bdf8-3430b0bf461f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.506162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9ea0-fc7f-4a41-bdf8-3430b0bf461f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.506394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9ea0-fc7f-4a41-bdf8-3430b0bf461f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.507764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 79/154] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:20.514029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eea719-4d97-4e2c-b89c-8cb9b70cc092 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.514313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eea719-4d97-4e2c-b89c-8cb9b70cc092 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.514588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eea719-4d97-4e2c-b89c-8cb9b70cc092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.520323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f86ff-a939-45ec-ab03-748a716de5e7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:20.536401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-36eea719-4d97-4e2c-b89c-8cb9b70cc092 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:20.537581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-36eea719-4d97-4e2c-b89c-8cb9b70cc092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:20.548311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:20.568420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:20.569071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.569351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.569629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] PATH_INFO: `/v3/users/7dab5b95f6784a02baf0bd1a2e4a2de6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.577110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] RBAC: Authorizing `identity:get_user(user_id=7dab5b95f6784a02baf0bd1a2e4a2de6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:20.578661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:20.580572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 76/155] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:20 2022] GET /identity/v3/users/7dab5b95f6784a02baf0bd1a2e4a2de6 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:20.586026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:20.613853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 80/156] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:20.732682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404edda2-71c6-47a2-8a54-b8210c0c3c48 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:20.761724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:20.771857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cae2b-03c6-4f95-88e2-6620e8de53ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:20.772522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cae2b-03c6-4f95-88e2-6620e8de53ca None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.772797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cae2b-03c6-4f95-88e2-6620e8de53ca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.773067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cae2b-03c6-4f95-88e2-6620e8de53ca None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.774227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cae2b-03c6-4f95-88e2-6620e8de53ca None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:20.775735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cae2b-03c6-4f95-88e2-6620e8de53ca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:20.777591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 77/157] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:20 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:20.783231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eea719-4d97-4e2c-b89c-8cb9b70cc092 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:20.893842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eea719-4d97-4e2c-b89c-8cb9b70cc092 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:20.935730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:20.936438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:20.936718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:20.936996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/7dab5b95f6784a02baf0bd1a2e4a2de6/roles/5615ab8f7bfd40e5a904031e70bcc62b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:20.938936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b71340f7ded406b9f92818f9e00a227, user_id=7dab5b95f6784a02baf0bd1a2e4a2de6, role_id=5615ab8f7bfd40e5a904031e70bcc62b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:20.940474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:20.964117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 81/158] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:20 2022] PUT /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227/users/7dab5b95f6784a02baf0bd1a2e4a2de6/roles/5615ab8f7bfd40e5a904031e70bcc62b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:21.110268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa51f6-939f-4dc0-8b7f-e2d34d75cab1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.111619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa51f6-939f-4dc0-8b7f-e2d34d75cab1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.112112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa51f6-939f-4dc0-8b7f-e2d34d75cab1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.114022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 78/159] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:21.114257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:21.119494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:21.131051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0724f42-fd20-4980-b7ff-766a3d0ff865 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.131663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0724f42-fd20-4980-b7ff-766a3d0ff865 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.132378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0724f42-fd20-4980-b7ff-766a3d0ff865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.160672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e0724f42-fd20-4980-b7ff-766a3d0ff865 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:21.162307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e0724f42-fd20-4980-b7ff-766a3d0ff865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:21.243074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0946d5-9b6f-4793-8903-1855503b8ef2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:21.246261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 79/160] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:21.247745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99dfb5-e089-4f3c-908b-75860805ef12 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.248271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99dfb5-e089-4f3c-908b-75860805ef12 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.248747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99dfb5-e089-4f3c-908b-75860805ef12 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.250270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 80/161] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:21.255710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b68e6c-44ca-49ca-b036-b4b4d88b3025 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.256282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b68e6c-44ca-49ca-b036-b4b4d88b3025 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.256775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b68e6c-44ca-49ca-b036-b4b4d88b3025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.288595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-78b68e6c-44ca-49ca-b036-b4b4d88b3025 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:21.290755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-78b68e6c-44ca-49ca-b036-b4b4d88b3025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:21.298622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:21.299273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.299551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.299810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.324024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:21.334327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:21.335237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:21.336104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 82/162] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:21.343953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:21.431827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:21.444908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 81/163] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:21.480485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74a21-f06c-4539-8284-27d35a690f2a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:21.519743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:21.520375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.520650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.520917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.521577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:21.521874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:21.523237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:21.532267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 83/164] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:21.537927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b68e6c-44ca-49ca-b036-b4b4d88b3025 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:21.621125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:21.678890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b68e6c-44ca-49ca-b036-b4b4d88b3025 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:21.722619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:21.724284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.725194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.726008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] PATH_INFO: `/v3/users/a8eb78c86bca426c9163deb9994b5cec` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.729145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] RBAC: Authorizing `identity:get_user(user_id=a8eb78c86bca426c9163deb9994b5cec)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:21.732308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:21.734868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 82/165] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:21 2022] GET /identity/v3/users/a8eb78c86bca426c9163deb9994b5cec => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:21.742418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:21.744311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:21.916126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1547d-c019-494c-9c71-157138dc6023 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:21.944067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:21.953887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:21.954925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:21.955318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:21.955935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:21.962053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c0dd-b854-4886-af84-909fd95be9ca None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:22.017038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:22.043606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:22.044883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:22.046047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:22.046754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:22.075555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:22.082995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:22.083894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:22.084954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:22.093584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 83/166] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:22.094432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:22.134896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:22.138015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:22.250983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff5d02-8c9b-43d0-ac69-612d591a3f60 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:22.291483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:22.292049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:22.292348 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:22.292620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:22.307962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:22.313036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:22.313911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:22.314668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 84/167] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:22.315406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:22.341263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 84/168] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:22.342154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:22.372598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:22.406953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:22.520668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb18b8-2259-4ea5-862c-6db679767c8a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:22.576520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8b206-a448-43b5-a03b-f06e71f3cefb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:22.614631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:22.615261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:22.615548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:22.615829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:22.616543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:22.616851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:22.618893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:22.636149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 85/169] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:22.637053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:22.661643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:22.662267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:22.662548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:22.662819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:22.663486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:22.663785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:22.665831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:22.683523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 85/170] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:22.684475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:22.820818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:22.833190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5543a2-6073-4a7a-b0c2-7fc7e75d1c8e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:22.847352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bf04-ca3d-424a-a380-2e3f5f456374 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:22.860559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:22.873683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:22.874629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:22.875126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:22.875623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:22.877459 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:22.879235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:22.881200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 86/171] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:22 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:22.882303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:22.927304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:22.927984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:22.928276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:22.928558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:22.931077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:22.932696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:22.934350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 86/172] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:22 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:22.935217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:22.995840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49855288-d1d2-4b38-bc22-2196f4ed3f36 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:23.055729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:23.056847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.057741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.058244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] PATH_INFO: `/v3/users/801bea5ab5ed40fd9053f989885b4314` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.060831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] RBAC: Authorizing `identity:get_user(user_id=801bea5ab5ed40fd9053f989885b4314)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:23.062492 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:23.064322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 87/173] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:22 2022] GET /identity/v3/users/801bea5ab5ed40fd9053f989885b4314 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:23.065444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:23.096687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:23.190555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3a783-b5f2-4473-8974-b0a73657cc25 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:23.192849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d913c27-6249-45f6-be04-bf1d7f5e31e4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:23.240973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:23.241594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.241880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.242193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.242942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=a8eb78c86bca426c9163deb9994b5cec) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:23.243249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:23.251233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:23.251530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 88/174] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:23 2022] GET /identity/v3/role_assignments?user.id=a8eb78c86bca426c9163deb9994b5cec&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:23.252394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:23.255341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad874a2-2137-49e3-8800-d729c4657210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:23.256540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad874a2-2137-49e3-8800-d729c4657210 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.256978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad874a2-2137-49e3-8800-d729c4657210 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.257431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad874a2-2137-49e3-8800-d729c4657210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.358848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:23.401372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad874a2-2137-49e3-8800-d729c4657210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:23.403174 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad874a2-2137-49e3-8800-d729c4657210 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:23.449068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:23.473098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3ea6-2221-4375-b540-13fcafadbd0e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:23.534562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a38992-48be-4b9b-b012-272044934193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:23.535191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38992-48be-4b9b-b012-272044934193 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.535473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38992-48be-4b9b-b012-272044934193 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.535750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38992-48be-4b9b-b012-272044934193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.573439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 87/175] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:23.574778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75326cc5-656b-4b69-b0a2-7f5d53c02a47 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.575078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75326cc5-656b-4b69-b0a2-7f5d53c02a47 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.575345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75326cc5-656b-4b69-b0a2-7f5d53c02a47 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.577797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 88/176] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:23.578868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c69bf-373d-4e04-8e8b-64603474d422 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.579291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c69bf-373d-4e04-8e8b-64603474d422 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.579546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c69bf-373d-4e04-8e8b-64603474d422 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.580926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 89/177] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:23.584005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c69bf-373d-4e04-8e8b-64603474d422 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:23.613955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a38992-48be-4b9b-b012-272044934193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:23.615331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a38992-48be-4b9b-b012-272044934193 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:23.674635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:23.716102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c69bf-373d-4e04-8e8b-64603474d422 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:23.718275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:23.733563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 89/178] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:23.735528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60c9c8-503e-4377-8553-736d612c3953 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.735528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60c9c8-503e-4377-8553-736d612c3953 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.740101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60c9c8-503e-4377-8553-736d612c3953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.760304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9d60c9c8-503e-4377-8553-736d612c3953 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:23.761513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9d60c9c8-503e-4377-8553-736d612c3953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:23.766215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158627db-1897-42fa-a103-21e2818cacf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:23.766844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158627db-1897-42fa-a103-21e2818cacf6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.767119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158627db-1897-42fa-a103-21e2818cacf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.767387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158627db-1897-42fa-a103-21e2818cacf6 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.768524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158627db-1897-42fa-a103-21e2818cacf6 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:23.769948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158627db-1897-42fa-a103-21e2818cacf6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:23.771752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 90/179] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:23 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:23.772750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5950cf-7d5e-4e31-8d2e-4a0ce6625167 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.773027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5950cf-7d5e-4e31-8d2e-4a0ce6625167 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.773325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5950cf-7d5e-4e31-8d2e-4a0ce6625167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.791875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9e5950cf-7d5e-4e31-8d2e-4a0ce6625167 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:23.793070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9e5950cf-7d5e-4e31-8d2e-4a0ce6625167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:23.833835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 90/180] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:23.835070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814c2a-f022-4f25-b69d-2ff6e5af2fdd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:23.835351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814c2a-f022-4f25-b69d-2ff6e5af2fdd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:23.835624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814c2a-f022-4f25-b69d-2ff6e5af2fdd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:23.836926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 91/181] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:23.840365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23814c2a-f022-4f25-b69d-2ff6e5af2fdd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:23.874042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 91/182] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:23.875067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5950cf-7d5e-4e31-8d2e-4a0ce6625167 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:23.951294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:23.963314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23814c2a-f022-4f25-b69d-2ff6e5af2fdd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:23.983746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5950cf-7d5e-4e31-8d2e-4a0ce6625167 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:23.984284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:24.009154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063edad5-40f3-46cb-8c1d-8a16178277dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.010237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063edad5-40f3-46cb-8c1d-8a16178277dd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.010727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063edad5-40f3-46cb-8c1d-8a16178277dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.011200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063edad5-40f3-46cb-8c1d-8a16178277dd None admin] PATH_INFO: `/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.012673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063edad5-40f3-46cb-8c1d-8a16178277dd None admin] RBAC: Authorizing `identity:get_project(project_id=3ddac4aad8804629b3ec37c0bdd42cb8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:24.014431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063edad5-40f3-46cb-8c1d-8a16178277dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:24.016686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 92/183] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:23 2022] GET /identity/v3/projects/3ddac4aad8804629b3ec37c0bdd42cb8 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:24.017992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8c4b7-1980-4850-8389-c7c243d0f391 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.018525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8c4b7-1980-4850-8389-c7c243d0f391 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.019027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8c4b7-1980-4850-8389-c7c243d0f391 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.020579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 93/184] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:24.025837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d4f4-5a75-489e-a7a2-e1bd94113ef6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.026158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d4f4-5a75-489e-a7a2-e1bd94113ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.026438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d4f4-5a75-489e-a7a2-e1bd94113ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.031431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.032399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.033171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.034638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.035624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] RBAC: Adding query filter params (role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=801bea5ab5ed40fd9053f989885b4314) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:24.036115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:24.037825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:24.045549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 92/185] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:23 2022] GET /identity/v3/role_assignments?user.id=801bea5ab5ed40fd9053f989885b4314&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=fdf7d3a219d340db99038e400383416b => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:24.046005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d795d4f4-5a75-489e-a7a2-e1bd94113ef6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:24.046788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.047565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d795d4f4-5a75-489e-a7a2-e1bd94113ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:24.175822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 94/186] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:24.175992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742020ed-eb26-461d-8d56-b01525536f5f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.177700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd10b-9359-4c83-8a82-289daceb8592 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.177991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd10b-9359-4c83-8a82-289daceb8592 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.178269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd10b-9359-4c83-8a82-289daceb8592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.198491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-523dd10b-9359-4c83-8a82-289daceb8592 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:24.199664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-523dd10b-9359-4c83-8a82-289daceb8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:24.218590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.219204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.219483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.219760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.220519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=3ddac4aad8804629b3ec37c0bdd42cb8, user.id=801bea5ab5ed40fd9053f989885b4314) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:24.220827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:24.222347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:24.226871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:24.240341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 93/187] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/role_assignments?user.id=801bea5ab5ed40fd9053f989885b4314&scope.project.id=3ddac4aad8804629b3ec37c0bdd42cb8&role.id=61a194f6aba541de8c32f53dfc572624 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:24.242461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.307409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 95/188] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:24.308686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd10b-9359-4c83-8a82-289daceb8592 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.370851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:24.439476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd10b-9359-4c83-8a82-289daceb8592 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.460590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1df90d-2181-4dc9-bbd4-be332ed30b1b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.488490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.489199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.489476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.490028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.503129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:24.510578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.511447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.511900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:24.512385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:24.513205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:24.514178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 96/189] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:24.515094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.515533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.528641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:24.534263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:24.535103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:24.535901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:24.537805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 94/190] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:24.539093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.604057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:24.622387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4937382-4e2b-4eba-8920-8cb5e515dd90 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.630323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:24.650291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdded1a6-980e-48e7-8012-c236e13d3dd4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.658860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.659499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.659777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.660105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.661272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:24.662711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:24.664359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 97/191] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:24.664359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.684319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.684953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.685224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.685491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.686149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:24.686447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:24.687798 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:24.696306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 95/192] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:24.700224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.764174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaa1b3-9e7e-470d-8e8f-f20328d8d7bb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.796101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf5b83-7593-4b97-9834-b6894ccaaa5e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.797557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.798143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.798428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.798699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.799358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:24.799656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:24.801047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:24.809299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 98/193] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:24.810092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.821215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:24.835293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.836534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.836845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.837120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.844733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:24.847287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:24.852594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:24.853586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:24.854345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 96/194] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:24.855036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.910982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df52f5d-d13c-4228-be15-9804447aa0bd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.943505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.944113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.944420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.944696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:24.945860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:24.947338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:24.954456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c541453-a4cb-4750-8bed-c44783ab6dec None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:24.974001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 99/195] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:42:24 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:24.974821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:24.993241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:24.993954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:24.994245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:24.994802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] PATH_INFO: `/v3/users/62e5a1491e064f3c8374526223750db2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.011513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] RBAC: Authorizing `identity:get_user(user_id=62e5a1491e064f3c8374526223750db2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.013161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.019777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 97/196] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/users/62e5a1491e064f3c8374526223750db2 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:25.019777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.067561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.081943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.096068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b656-096b-4dfb-96ef-cabf68051769 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.137401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.138039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.138315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.138517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8333c-61e4-4ad4-92e4-ff05b08fa473 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.138627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] PATH_INFO: `/v3/users/62e5a1491e064f3c8374526223750db2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.139701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] RBAC: Authorizing `identity:get_user(user_id=62e5a1491e064f3c8374526223750db2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.141431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.142826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 100/197] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:24 2022] GET /identity/v3/users/62e5a1491e064f3c8374526223750db2 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:25.143587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.178322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.178928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.179205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.179984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.180681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:25.180996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.182362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.192288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 98/198] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:25.192852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.275421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a6ff-227b-4746-9729-26bf0f165741 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.315640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.318051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.318672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.318947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.319386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.320025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b5400-2cbc-4b0a-b9c8-c7a619286c7e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.361213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.369619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.370521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.370794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.371980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.417758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.419333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.457322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.458806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.541734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 101/199] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:25.542498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.567466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.576687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 99/200] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:25.578221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.621293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.678581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde75c72-b039-4495-b4a4-41b4c1e1a537 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.691499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab81aa-eb3e-4c0f-90a0-b2efc48b5b7b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.723255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.724200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.724507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.724787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] PATH_INFO: `/v3/users/7dab5b95f6784a02baf0bd1a2e4a2de6` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.725950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] RBAC: Authorizing `identity:get_user(user_id=7dab5b95f6784a02baf0bd1a2e4a2de6)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.727935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.728621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.729577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.729866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 102/201] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/users/7dab5b95f6784a02baf0bd1a2e4a2de6 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:25.730145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.730516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.730746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.731870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.733486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.735443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 100/202] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:25.736475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.826007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.853596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e745f6c-3763-4c77-a2ea-eab0567b8e84 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.870299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf6cfe-bcf2-4440-96f4-191a54e348ef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:25.886884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:25.900180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.900842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.901544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.901832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:25.903206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:25.905338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:25.907809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 103/203] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:25.908612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:25.918747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:25.919544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:25.919920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:25.920381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:26.001096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:26.003017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:26.026367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b277ed-6a72-4891-87aa-c2073a4ab226 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:26.070834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:26.071442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:26.071722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:26.072279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:26.073418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] RBAC: Adding query filter params (role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=62e5a1491e064f3c8374526223750db2) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:26.073728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:26.075680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:26.081298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:26.083884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 104/204] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/role_assignments?user.id=62e5a1491e064f3c8374526223750db2&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=fdf7d3a219d340db99038e400383416b => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:26.084846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:26.123549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 101/205] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:26.129684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:26.187760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:26.254532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf105c0-ee43-46d5-9a9f-d4ea284f1c31 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:26.259442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af2bf0-d4fb-4b38-879e-8f02722313e9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:26.308457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:26.309076 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:26.309465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:26.309745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:26.310655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=62e5a1491e064f3c8374526223750db2) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:26.310894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:26.312378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:26.315668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d60650-9c9c-4bc5-89bb-8d5331b948c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:26.316596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d60650-9c9c-4bc5-89bb-8d5331b948c2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:26.317045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d60650-9c9c-4bc5-89bb-8d5331b948c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:26.317459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d60650-9c9c-4bc5-89bb-8d5331b948c2 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:26.318367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 105/206] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:26 2022] GET /identity/v3/role_assignments?user.id=62e5a1491e064f3c8374526223750db2&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:26.318915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d60650-9c9c-4bc5-89bb-8d5331b948c2 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:26.321511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d60650-9c9c-4bc5-89bb-8d5331b948c2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:26.322618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 102/207] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:26 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:26.329718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:26.464417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed137da-74e9-4263-a344-ec158f1d311b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:26.491071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:26.513953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6eb559-c848-4dcc-a909-4a1d050f847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:26.514978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eb559-c848-4dcc-a909-4a1d050f847c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:26.515513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eb559-c848-4dcc-a909-4a1d050f847c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:26.516016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eb559-c848-4dcc-a909-4a1d050f847c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:26.517085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6eb559-c848-4dcc-a909-4a1d050f847c None admin] RBAC: Adding query filter params (role.id=5615ab8f7bfd40e5a904031e70bcc62b, scope.project.id=8b71340f7ded406b9f92818f9e00a227, user.id=7dab5b95f6784a02baf0bd1a2e4a2de6) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:26.517604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6eb559-c848-4dcc-a909-4a1d050f847c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:26.519282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6eb559-c848-4dcc-a909-4a1d050f847c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:26.530109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 106/208] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:26 2022] GET /identity/v3/role_assignments?user.id=7dab5b95f6784a02baf0bd1a2e4a2de6&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=5615ab8f7bfd40e5a904031e70bcc62b => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:27.192343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aec28-3fa9-41e6-af89-3d3a60c9aaf9 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.192880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aec28-3fa9-41e6-af89-3d3a60c9aaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.192880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aec28-3fa9-41e6-af89-3d3a60c9aaf9 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.194374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 103/209] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:27.198240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78fb7-35ca-4e29-a60d-fb8f5fa1b76a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.198539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78fb7-35ca-4e29-a60d-fb8f5fa1b76a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.198817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78fb7-35ca-4e29-a60d-fb8f5fa1b76a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.200302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 107/210] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:27.201484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4be0-0b89-4349-aa32-a833a54594a5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.201765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4be0-0b89-4349-aa32-a833a54594a5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.202038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e4be0-0b89-4349-aa32-a833a54594a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.211887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbcfb-ffb2-492a-b887-23fcfef80f91 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.212172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbcfb-ffb2-492a-b887-23fcfef80f91 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.212496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadbcfb-ffb2-492a-b887-23fcfef80f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.222806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fe4e4be0-0b89-4349-aa32-a833a54594a5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:27.224316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fe4e4be0-0b89-4349-aa32-a833a54594a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:27.236271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0dadbcfb-ffb2-492a-b887-23fcfef80f91 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:27.236271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0dadbcfb-ffb2-492a-b887-23fcfef80f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:27.255018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:27.299558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 104/211] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:27.316461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 108/212] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:27.575080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4be0-0b89-4349-aa32-a833a54594a5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:27.584396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadbcfb-ffb2-492a-b887-23fcfef80f91 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:27.687014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e4be0-0b89-4349-aa32-a833a54594a5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:27.721037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadbcfb-ffb2-492a-b887-23fcfef80f91 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:27.723149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:27.724015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.728286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.728286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.737405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:27.742384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:27.743443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:27.744458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:27.750182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 105/213] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:27.750471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:27.751167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:27.767035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:27.767955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.768453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.768903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.770392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:27.771997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:27.774883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 109/214] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:27.784354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:27.784492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:27.865656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52d1fa-8466-4b45-b135-98ec55d630c8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:27.905188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:27.905927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.906452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.906726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.907500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:27.907875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:27.909303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:27.916126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd31583-e068-4793-b841-d6b39a1625f4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:27.917785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 106/215] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:27.923156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:27.953924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:27.955053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:27.955536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:27.956007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:27.956902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:27.958643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:27.986081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 110/216] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:42:27 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:28.037389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:28.048867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd706f3-3e4d-4693-ab73-d662d37e6747 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:28.088624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febae9d5-0c77-4e87-a628-4f8ae1ed3fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:28.089446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febae9d5-0c77-4e87-a628-4f8ae1ed3fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.089824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febae9d5-0c77-4e87-a628-4f8ae1ed3fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.090252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febae9d5-0c77-4e87-a628-4f8ae1ed3fb2 None admin] PATH_INFO: `/v3/users/154812ddd98748e4b36746123ff5e64f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.106524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febae9d5-0c77-4e87-a628-4f8ae1ed3fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=154812ddd98748e4b36746123ff5e64f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:28.108175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febae9d5-0c77-4e87-a628-4f8ae1ed3fb2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:28.110071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 107/217] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:27 2022] GET /identity/v3/users/154812ddd98748e4b36746123ff5e64f => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:28.116203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:28.164615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:28.227694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f65c8-ba85-4d71-a0d8-f8b83d13d030 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:28.264488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:28.265108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.265388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.265662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.340386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:28.341876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:28.351555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de258b57-5307-4668-ac77-e34554c4ffd9 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.352051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de258b57-5307-4668-ac77-e34554c4ffd9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.352127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de258b57-5307-4668-ac77-e34554c4ffd9 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.353735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 108/218] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:28.359410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8dba2-6389-4403-a27c-a3f7c5c64d79 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.359698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8dba2-6389-4403-a27c-a3f7c5c64d79 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.359974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8dba2-6389-4403-a27c-a3f7c5c64d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.389352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-37c8dba2-6389-4403-a27c-a3f7c5c64d79 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:28.391237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-37c8dba2-6389-4403-a27c-a3f7c5c64d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:28.391346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:28.471706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 111/219] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:28.478908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:28.489251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 109/220] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:28.490655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab5f1-aa2d-409c-80d9-cdf487061b4a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.490942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab5f1-aa2d-409c-80d9-cdf487061b4a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.491337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab5f1-aa2d-409c-80d9-cdf487061b4a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.493294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 110/221] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:28.498259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b519ea-6b42-4b3b-9719-417e9f9debee None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.498622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b519ea-6b42-4b3b-9719-417e9f9debee None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.498966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b519ea-6b42-4b3b-9719-417e9f9debee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.519131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-51b519ea-6b42-4b3b-9719-417e9f9debee None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:28.520440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-51b519ea-6b42-4b3b-9719-417e9f9debee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:28.621095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b87db1-c8f2-4ef7-953a-e469d9e2fded None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:28.627200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 111/222] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:28.675005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:28.675566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.675839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.676109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.677289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:28.678818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:28.681267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 112/223] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:28 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:28.687091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b519ea-6b42-4b3b-9719-417e9f9debee None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:28.695753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:28.789311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:28.803125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:28.820487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b519ea-6b42-4b3b-9719-417e9f9debee None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:28.884385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:28.884999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:28.885282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:28.885553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:28.886288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=154812ddd98748e4b36746123ff5e64f) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:28.886592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:28.889173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:28.901401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 112/224] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:28 2022] GET /identity/v3/role_assignments?user.id=154812ddd98748e4b36746123ff5e64f&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=61a194f6aba541de8c32f53dfc572624 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:28.940553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc3e9-2040-4490-a71c-d81898b8d130 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:28.978711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.004207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.004945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.005225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.005549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.027464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:29.032953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:29.033893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:29.034710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 113/225] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:29.041335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.088619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:29.119699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812752-a9b4-42db-b5b4-2dbf6bb49844 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:29.136141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:29.170390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0d6b6-dee2-43b5-80ca-5690837e724c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:29.201088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.202078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.202840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.203564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.210738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.211603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.211996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.212395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.213433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:29.213829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:29.215559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:29.216036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:29.223629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:29.224712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:29.225486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:29.226083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 114/226] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:29.227488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 113/227] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:29.232823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.233449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.383939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab05e-934f-4d61-8486-aee708ff7620 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:29.402632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce120e46-2356-455e-b7fc-c19b3124e914 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:29.434978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:29.441653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.442588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.443153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.443647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] PATH_INFO: `/v3/users/62e5a1491e064f3c8374526223750db2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.445184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] RBAC: Authorizing `identity:get_user(user_id=62e5a1491e064f3c8374526223750db2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:29.447085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:29.449413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 115/228] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:29 2022] GET /identity/v3/users/62e5a1491e064f3c8374526223750db2 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:29.455552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.461270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:29.469064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.469686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.470039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.470297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.470971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:29.471273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:29.473548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:29.494615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 114/229] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:29.505304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.579525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa015b97-d3f6-4a1f-aac0-0ac3be1b4312 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:29.618737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.619422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.619708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.619982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.621274 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:29.622776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:29.624762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 116/230] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:29 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:29.629989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.737384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:29.738568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c745fed-d4af-4353-82f4-cb71338f8565 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:29.774341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642b631-9f5a-4e13-85cb-d6e00a9aed86 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:29.807617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.808389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.808674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.808942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] PATH_INFO: `/v3/users/62e5a1491e064f3c8374526223750db2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.810121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] RBAC: Authorizing `identity:get_user(user_id=62e5a1491e064f3c8374526223750db2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:29.811631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:29.811753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:29.812647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:29.813429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:29.813602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 115/231] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:29 2022] GET /identity/v3/users/62e5a1491e064f3c8374526223750db2 => generated 283 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:29.814129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/62e5a1491e064f3c8374526223750db2/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:29.822970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:29.823485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d69e08be64483acd3b1c8fd75c744, user_id=62e5a1491e064f3c8374526223750db2, role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:29.825852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:29.849770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 117/232] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:29 2022] PUT /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/62e5a1491e064f3c8374526223750db2/roles/fdf7d3a219d340db99038e400383416b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:29.935308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:30.001016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b22785-1d90-4dac-85b3-729b707888f4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:30.040827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd0449-ec5c-4a39-b3ca-f42516a1317f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:30.041467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd0449-ec5c-4a39-b3ca-f42516a1317f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:30.041742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd0449-ec5c-4a39-b3ca-f42516a1317f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:30.042007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd0449-ec5c-4a39-b3ca-f42516a1317f None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:30.043117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd0449-ec5c-4a39-b3ca-f42516a1317f None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:30.044599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd0449-ec5c-4a39-b3ca-f42516a1317f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:30.046384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 116/233] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:29 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:30.052406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:30.196411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cb444-e588-4e60-bc25-124b6b03bfdb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:30.223665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:30.233214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6faa4e-667e-4269-abf1-2e7eaaf7caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:30.233889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6faa4e-667e-4269-abf1-2e7eaaf7caa9 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:30.234222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6faa4e-667e-4269-abf1-2e7eaaf7caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:30.234500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6faa4e-667e-4269-abf1-2e7eaaf7caa9 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/62e5a1491e064f3c8374526223750db2/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:30.236909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6faa4e-667e-4269-abf1-2e7eaaf7caa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d69e08be64483acd3b1c8fd75c744, user_id=62e5a1491e064f3c8374526223750db2, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:30.239013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6faa4e-667e-4269-abf1-2e7eaaf7caa9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:30.259836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 118/234] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:30 2022] PUT /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/62e5a1491e064f3c8374526223750db2/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:30.511433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287659f-1ae5-4307-bfb6-d2ba2814592b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:30.511908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287659f-1ae5-4307-bfb6-d2ba2814592b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:30.511908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287659f-1ae5-4307-bfb6-d2ba2814592b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:30.513511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 117/235] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:30.519051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912ec87-8162-4ed7-b6ff-fcd6ae04b615 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:30.519445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912ec87-8162-4ed7-b6ff-fcd6ae04b615 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:30.519607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912ec87-8162-4ed7-b6ff-fcd6ae04b615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:30.541266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9912ec87-8162-4ed7-b6ff-fcd6ae04b615 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:30.542450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9912ec87-8162-4ed7-b6ff-fcd6ae04b615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:30.623529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 119/236] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:30.861498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907d326-abc0-484e-8733-e1ecb8080e82 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:30.861831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907d326-abc0-484e-8733-e1ecb8080e82 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:30.862019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907d326-abc0-484e-8733-e1ecb8080e82 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:30.863297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 118/237] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:30.868717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41895588-d2b9-4306-92ea-05ba500fdaae None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:30.869005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41895588-d2b9-4306-92ea-05ba500fdaae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:30.869285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41895588-d2b9-4306-92ea-05ba500fdaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:30.887802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-41895588-d2b9-4306-92ea-05ba500fdaae None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:30.889028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-41895588-d2b9-4306-92ea-05ba500fdaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:30.935216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907d326-abc0-484e-8733-e1ecb8080e82 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:30.995529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 120/238] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:31.068757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907d326-abc0-484e-8733-e1ecb8080e82 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:31.112133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:31.125805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedbfd9-d38d-47e0-867a-af8d0e26b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:31.126444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedbfd9-d38d-47e0-867a-af8d0e26b34c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:31.126722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedbfd9-d38d-47e0-867a-af8d0e26b34c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:31.126994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedbfd9-d38d-47e0-867a-af8d0e26b34c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:31.128108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bedbfd9-d38d-47e0-867a-af8d0e26b34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:31.129575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bedbfd9-d38d-47e0-867a-af8d0e26b34c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:31.131293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 119/239] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:31.137211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41895588-d2b9-4306-92ea-05ba500fdaae None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:31.200973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:31.270316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41895588-d2b9-4306-92ea-05ba500fdaae None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:31.331277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:31.331904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:31.332176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:31.332487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:31.333134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:31.334551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:31.377336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3453f0-6299-4d72-bfbf-521ee4a60bb7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:31.377613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3453f0-6299-4d72-bfbf-521ee4a60bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:31.377822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3453f0-6299-4d72-bfbf-521ee4a60bb7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:31.379389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 120/240] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:31.384617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32843b02-3a2f-4467-9ef8-fe2ba4b49c67 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:31.384901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32843b02-3a2f-4467-9ef8-fe2ba4b49c67 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:31.385173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32843b02-3a2f-4467-9ef8-fe2ba4b49c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:31.389650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 121/241] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:42:31 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:42:31.421447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-32843b02-3a2f-4467-9ef8-fe2ba4b49c67 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:31.422691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-32843b02-3a2f-4467-9ef8-fe2ba4b49c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:31.431156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:31.558744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7541742-17e8-4ad3-a31b-7efe2a41c426 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:31.561570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:31.565603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 121/242] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:31.596344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:31.597334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:31.597817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:31.598270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:31.610114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:31.616135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:31.617570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:31.618336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:31.620660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 122/243] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:31.626480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32843b02-3a2f-4467-9ef8-fe2ba4b49c67 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:31.730519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32843b02-3a2f-4467-9ef8-fe2ba4b49c67 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:31.758607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:31.768758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:31.769391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:31.770026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:31.770299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:31.770975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:31.771651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:31.773466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:31.782725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 122/244] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:31.788073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:31.882571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:31.902354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f128545-af60-4ac3-936b-5b5fb042a9d3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:31.957826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:31.958958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:31.959244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:31.959524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] PATH_INFO: `/v3/users/cd7402dc049c40a1a1a85b7ffad5f979` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:31.979591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] RBAC: Authorizing `identity:get_user(user_id=cd7402dc049c40a1a1a85b7ffad5f979)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:31.983330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:31.983330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 123/245] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:31 2022] GET /identity/v3/users/cd7402dc049c40a1a1a85b7ffad5f979 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:31.988896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.016484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f6fcb-0d63-447c-be67-91a55d11dc58 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.044937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:32.065751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164cfa5-e42b-4265-b063-b0e2ef6d2fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.066864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cfa5-e42b-4265-b063-b0e2ef6d2fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.067392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cfa5-e42b-4265-b063-b0e2ef6d2fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.067859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cfa5-e42b-4265-b063-b0e2ef6d2fc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.081564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164cfa5-e42b-4265-b063-b0e2ef6d2fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e164cfa5-e42b-4265-b063-b0e2ef6d2fc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:32.088089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:32.089025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:32.089806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:32.091779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 123/246] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:32.093238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b8ec-701b-4a9f-9f2e-1f2ae365e0d7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.093730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b8ec-701b-4a9f-9f2e-1f2ae365e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.094203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01b8ec-701b-4a9f-9f2e-1f2ae365e0d7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.096089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 124/247] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:32.101082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b236cc-8017-44a1-bbea-57b78cfbdf49 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.102279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b236cc-8017-44a1-bbea-57b78cfbdf49 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.102881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b236cc-8017-44a1-bbea-57b78cfbdf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.113433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94340d63-87f0-4301-b574-2d6e4b1a3e5b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.125024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-68b236cc-8017-44a1-bbea-57b78cfbdf49 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:32.126819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-68b236cc-8017-44a1-bbea-57b78cfbdf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:32.154197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.154846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.155100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.155375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.180510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:32.215785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 125/248] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:32.217055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b236cc-8017-44a1-bbea-57b78cfbdf49 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.236980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:32.238637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:32.264597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:32.368563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 124/249] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:32.373319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.375805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b236cc-8017-44a1-bbea-57b78cfbdf49 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.443164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.443785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.444058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.444392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.446332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:32.446635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:32.448007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:32.465135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 126/250] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:32.470303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.555194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed65f3-f22e-4768-bbb7-192d66791062 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.555824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:32.592635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:32.602697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.603346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.603630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.603908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.605084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:32.606548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:32.612529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 125/251] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:32 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:32.613418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.621206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee25f18-308f-41e9-8a01-2db407f345a2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.665729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.666438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.666692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.666979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] PATH_INFO: `/v3/users/154812ddd98748e4b36746123ff5e64f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.668093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] RBAC: Authorizing `identity:get_user(user_id=154812ddd98748e4b36746123ff5e64f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:32.669594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:32.671748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 127/252] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:32 2022] GET /identity/v3/users/154812ddd98748e4b36746123ff5e64f => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:32.672618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.764782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c0540-26d4-4209-9b7e-d9e119898917 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.782935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069e6fe-a9e9-4a5f-803c-b302022b32d4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.792392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:32.801272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.801909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.802182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.802446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.813549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:32.818387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:32.819241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:32.820600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 126/253] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:32.822411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.825434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454363a-33b5-4ce2-94f2-a724a36f165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.825434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454363a-33b5-4ce2-94f2-a724a36f165e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.825434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454363a-33b5-4ce2-94f2-a724a36f165e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.825434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454363a-33b5-4ce2-94f2-a724a36f165e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.826598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454363a-33b5-4ce2-94f2-a724a36f165e None admin] RBAC: Adding query filter params (role.id=5615ab8f7bfd40e5a904031e70bcc62b, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=cd7402dc049c40a1a1a85b7ffad5f979) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:32.827069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454363a-33b5-4ce2-94f2-a724a36f165e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:32.829462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454363a-33b5-4ce2-94f2-a724a36f165e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:32.837786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 128/254] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:32 2022] GET /identity/v3/role_assignments?user.id=cd7402dc049c40a1a1a85b7ffad5f979&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=5615ab8f7bfd40e5a904031e70bcc62b => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:32.839747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053742-a3ac-46e9-92ba-ecb4fd2e8b33 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.840154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053742-a3ac-46e9-92ba-ecb4fd2e8b33 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.840543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44053742-a3ac-46e9-92ba-ecb4fd2e8b33 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.842255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 129/255] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:32.843059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44053742-a3ac-46e9-92ba-ecb4fd2e8b33 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:32.882313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:32.935858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a40e-2f43-48e0-85ec-c2fe8948225e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.959920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44053742-a3ac-46e9-92ba-ecb4fd2e8b33 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:32.971705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8ba0e-6bad-4c60-922b-455e0d2421b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.972407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8ba0e-6bad-4c60-922b-455e0d2421b7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.972685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8ba0e-6bad-4c60-922b-455e0d2421b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.972957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8ba0e-6bad-4c60-922b-455e0d2421b7 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.974071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8ba0e-6bad-4c60-922b-455e0d2421b7 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:32.975510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8ba0e-6bad-4c60-922b-455e0d2421b7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:32.977437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 127/256] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:32 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:32.978404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef232-8a60-4113-a12a-e6fd57e43e66 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.978685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef232-8a60-4113-a12a-e6fd57e43e66 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.978959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef232-8a60-4113-a12a-e6fd57e43e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:32.998368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:32.998976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:32.999295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:32.999616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:33.000301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:33.000651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:33.002238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:33.003417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c06ef232-8a60-4113-a12a-e6fd57e43e66 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:33.004661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c06ef232-8a60-4113-a12a-e6fd57e43e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:33.012145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 130/257] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:33.013121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:33.050491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:33.075040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 128/258] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:33.075833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef232-8a60-4113-a12a-e6fd57e43e66 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:33.125286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7863573-a86a-48bd-9b2d-d837a9a9f722 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:33.156651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:33.168116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:33.169049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:33.169385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:33.169598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/154812ddd98748e4b36746123ff5e64f/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:33.173766 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d69e08be64483acd3b1c8fd75c744, user_id=154812ddd98748e4b36746123ff5e64f, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:33.175212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:33.211320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 131/259] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:33 2022] PUT /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/154812ddd98748e4b36746123ff5e64f/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:33.230609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef232-8a60-4113-a12a-e6fd57e43e66 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:33.288329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:33.289534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:33.289813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:33.290081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] PATH_INFO: `/v3/users/62e5a1491e064f3c8374526223750db2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:33.295227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] RBAC: Authorizing `identity:get_user(user_id=62e5a1491e064f3c8374526223750db2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:33.298350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:33.298568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 129/260] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:33 2022] GET /identity/v3/users/62e5a1491e064f3c8374526223750db2 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:33.304441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:33.410240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:33.417504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eef691-a40c-4f06-ab1c-333490684cc7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:33.454214 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:33.454851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:33.455128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:33.455401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:33.532950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:33.534409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:33.581415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:33.645834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:33.663465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 132/261] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:33.672315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:33.739688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b555-e4a5-4e0f-9035-d7757537f7ad None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:33.757819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:33.780945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:33.781925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:33.788564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:33.789654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:33.805355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:33.816268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:33.817152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:33.817918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:33.821257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 130/262] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:33.826741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:33.827831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42c631-3cb2-4acf-9f9b-6a220b64473a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:33.864368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:33.865011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:33.865338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:33.865616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:33.866914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:33.868529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:33.870165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 133/263] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:33 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:33.876966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:33.985165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:34.028909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50ba74-77d4-4ddb-bfc6-654ed88f5954 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:34.061948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a81cc-cfdc-4a82-b635-bef6d56f5da7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:34.096381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:34.098006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.098282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.098549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.102155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:34.102464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:34.103853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:34.125266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 131/264] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:34.128424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:34.137017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:34.137641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.137919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.138185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.138921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] RBAC: Adding query filter params (role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=62e5a1491e064f3c8374526223750db2) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:34.139223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:34.144794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:34.155235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 134/265] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:33 2022] GET /identity/v3/role_assignments?user.id=62e5a1491e064f3c8374526223750db2&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=fdf7d3a219d340db99038e400383416b => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:34.199735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:34.308741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216db6f-f320-46bb-ada7-8e948327266e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:34.359860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:34.360660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.361192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.361465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] PATH_INFO: `/v3/users/62e5a1491e064f3c8374526223750db2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.365030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] RBAC: Authorizing `identity:get_user(user_id=62e5a1491e064f3c8374526223750db2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:34.366500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:34.368226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 132/266] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:34 2022] GET /identity/v3/users/62e5a1491e064f3c8374526223750db2 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:34.374112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:34.513919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9002a-27f9-4103-ba4e-75cbfbc90c76 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:34.543426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:34.552512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6ba3f-911e-468f-a389-6698a2bafb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:34.553308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6ba3f-911e-468f-a389-6698a2bafb25 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.553580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6ba3f-911e-468f-a389-6698a2bafb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.553850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6ba3f-911e-468f-a389-6698a2bafb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.634281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6ba3f-911e-468f-a389-6698a2bafb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:34.635768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6ba3f-911e-468f-a389-6698a2bafb25 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:34.766336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 135/267] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:34.771774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:34.794792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361eb81-be4e-43a8-9476-ffb1e6843a5a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.795127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361eb81-be4e-43a8-9476-ffb1e6843a5a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.795337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361eb81-be4e-43a8-9476-ffb1e6843a5a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.796961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 136/268] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:34.801989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e495a7-5aad-4c24-9f6a-dab98c671bd2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.802275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e495a7-5aad-4c24-9f6a-dab98c671bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.802604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e495a7-5aad-4c24-9f6a-dab98c671bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.829357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-40e495a7-5aad-4c24-9f6a-dab98c671bd2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:34.830682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40e495a7-5aad-4c24-9f6a-dab98c671bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:34.856446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:34.909332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e7822-566f-42a4-b43e-d41705833c36 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:34.917360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 137/269] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:34.917519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:34.919681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8c4a-5774-4887-9df1-01d8f463eb29 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.920025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8c4a-5774-4887-9df1-01d8f463eb29 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.920347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8c4a-5774-4887-9df1-01d8f463eb29 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.921873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 138/270] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:34.927678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd1ed-4541-4c17-af8c-2baaae736712 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.928081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd1ed-4541-4c17-af8c-2baaae736712 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.929071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd1ed-4541-4c17-af8c-2baaae736712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.954218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:34.954853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:34.955127 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:34.955397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:34.955766 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2d4dd1ed-4541-4c17-af8c-2baaae736712 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:34.957034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2d4dd1ed-4541-4c17-af8c-2baaae736712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:34.957521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:34.959233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:34.962070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 133/271] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:34 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:34.968210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:35.077421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 139/272] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:35.129522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd627c2-1d1c-4ae8-a220-b82cd10237ba None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:35.177978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eee293-a92a-430e-b732-3666741db582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:35.178580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eee293-a92a-430e-b732-3666741db582 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.178857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eee293-a92a-430e-b732-3666741db582 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.179128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eee293-a92a-430e-b732-3666741db582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.179876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eee293-a92a-430e-b732-3666741db582 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=62e5a1491e064f3c8374526223750db2) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:35.180177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eee293-a92a-430e-b732-3666741db582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:35.181624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eee293-a92a-430e-b732-3666741db582 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:35.192980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 134/273] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:34 2022] GET /identity/v3/role_assignments?user.id=62e5a1491e064f3c8374526223750db2&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:35.225904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd1ed-4541-4c17-af8c-2baaae736712 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:35.341299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:35.361062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd1ed-4541-4c17-af8c-2baaae736712 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:35.406728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:35.407779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.408315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.408784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.419608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:35.425010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:35.425938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:35.426707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:35.429557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 140/274] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:35.435590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eee293-a92a-430e-b732-3666741db582 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:35.498483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:35.501610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:35.576840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eee293-a92a-430e-b732-3666741db582 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:35.615139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:35.615656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.615934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.616206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.616952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:35.617255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:35.618633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:35.627786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 135/275] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:35.633877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:35.653616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291a0ef-b091-4ca0-8087-d4972c0e1a58 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:35.703695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c670c-cb79-47c1-8038-958ff762a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:35.704378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c670c-cb79-47c1-8038-958ff762a8cf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.704655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c670c-cb79-47c1-8038-958ff762a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.704938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c670c-cb79-47c1-8038-958ff762a8cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.706139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c670c-cb79-47c1-8038-958ff762a8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:35.707893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c670c-cb79-47c1-8038-958ff762a8cf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:35.709685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 141/276] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:35.710680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191136e-6209-4f74-8253-eb192e491246 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.711025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191136e-6209-4f74-8253-eb192e491246 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.711301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191136e-6209-4f74-8253-eb192e491246 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.712652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 142/277] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:35.717145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191136e-6209-4f74-8253-eb192e491246 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:35.736521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:35.751385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:35.754637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dedc8c-af1d-4741-899c-c1e8e196b977 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:35.790339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573d276-164e-4679-8611-508dd09b7b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:35.791067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573d276-164e-4679-8611-508dd09b7b3b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.791338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573d276-164e-4679-8611-508dd09b7b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.791599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573d276-164e-4679-8611-508dd09b7b3b None admin] PATH_INFO: `/v3/users/14c01ae5f2fd4c44bd3388476d8a3b9e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.807233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573d276-164e-4679-8611-508dd09b7b3b None admin] RBAC: Authorizing `identity:get_user(user_id=14c01ae5f2fd4c44bd3388476d8a3b9e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:35.809019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573d276-164e-4679-8611-508dd09b7b3b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:35.815915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 136/278] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:35 2022] GET /identity/v3/users/14c01ae5f2fd4c44bd3388476d8a3b9e => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:35.815915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b4cfa-43be-4de7-82af-7118b20f99e9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.815915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b4cfa-43be-4de7-82af-7118b20f99e9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.815915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b4cfa-43be-4de7-82af-7118b20f99e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.833565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7d2b4cfa-43be-4de7-82af-7118b20f99e9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:35.833861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191136e-6209-4f74-8253-eb192e491246 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:35.834787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7d2b4cfa-43be-4de7-82af-7118b20f99e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:35.869593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:35.870594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:35.870594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:35.870760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:35.871292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:35.872755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:35.899339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 143/279] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:42:35 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:42:35.900082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:35.923291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 137/280] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:35.989971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:36.030079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d6143-10b0-4603-9437-715a14d2bec2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:36.068103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e94d7-a07f-4111-93b1-a2747bb325cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:36.069089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e94d7-a07f-4111-93b1-a2747bb325cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.069372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e94d7-a07f-4111-93b1-a2747bb325cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.069645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e94d7-a07f-4111-93b1-a2747bb325cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.070408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e94d7-a07f-4111-93b1-a2747bb325cc None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, user.id=14c01ae5f2fd4c44bd3388476d8a3b9e, scope.system=all) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:36.070717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e94d7-a07f-4111-93b1-a2747bb325cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:36.072162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e94d7-a07f-4111-93b1-a2747bb325cc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:36.077665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 144/281] 10.222.0.50 () {56 vars in 1308 bytes} [Tue Aug 30 01:42:35 2022] GET /identity/v3/role_assignments?user.id=14c01ae5f2fd4c44bd3388476d8a3b9e&scope.system=all&role.id=61a194f6aba541de8c32f53dfc572624 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:36.081808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bed3f9-0cb9-46fd-8862-e6e4b8eb91c8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.082100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bed3f9-0cb9-46fd-8862-e6e4b8eb91c8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.082376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bed3f9-0cb9-46fd-8862-e6e4b8eb91c8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.083937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 138/282] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:36.089584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dea80-0222-41ab-b82d-d4ae7213dbc8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.089888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dea80-0222-41ab-b82d-d4ae7213dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.090151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dea80-0222-41ab-b82d-d4ae7213dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.111238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-444dea80-0222-41ab-b82d-d4ae7213dbc8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:36.112848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-444dea80-0222-41ab-b82d-d4ae7213dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:36.190247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 145/283] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:36.198468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bed3f9-0cb9-46fd-8862-e6e4b8eb91c8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:36.318124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:36.337099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bed3f9-0cb9-46fd-8862-e6e4b8eb91c8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:36.381641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:36.382275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.382636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.382910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.399871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:36.404870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:36.405711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:36.406470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:36.407048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 139/284] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:36.411627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dea80-0222-41ab-b82d-d4ae7213dbc8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:36.507872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:36.533481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:36.579155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dea80-0222-41ab-b82d-d4ae7213dbc8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:36.637829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:36.638449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.638730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.639164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.639735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:36.639995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:36.641379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:36.655372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 146/285] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:36.660760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:36.679615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43bb5c-057e-4783-8fbe-372163cb8b97 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:36.718473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:36.719487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.719964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.720432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.732670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:36.738636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:36.739497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:36.740265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:36.743044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 140/286] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:36.749213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:36.761320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:36.860591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b212a00-0358-4474-85a4-1fd613a2d276 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:36.891783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c603512-f7f0-4b15-9f67-8c1c98fb05ce None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:36.897681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:36.899059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.899512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.899932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.900720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:36.901116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:36.902669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:36.911872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 141/287] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:36.917762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:36.924225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:36.932821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:36.933460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:36.933739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:36.934012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] PATH_INFO: `/v3/users/cd7402dc049c40a1a1a85b7ffad5f979` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:36.935154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] RBAC: Authorizing `identity:get_user(user_id=cd7402dc049c40a1a1a85b7ffad5f979)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:36.936630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:36.938658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 147/288] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:36 2022] GET /identity/v3/users/cd7402dc049c40a1a1a85b7ffad5f979 => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:36.949319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:37.004029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:37.030784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936002-4e02-4370-8a8c-9ddfa6e68f75 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:37.084987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:37.086265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:37.086540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:37.086809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] PATH_INFO: `/v3/users/154812ddd98748e4b36746123ff5e64f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:37.088068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] RBAC: Authorizing `identity:get_user(user_id=154812ddd98748e4b36746123ff5e64f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:37.089520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:37.091297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 142/289] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:36 2022] GET /identity/v3/users/154812ddd98748e4b36746123ff5e64f => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:37.101378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:37.187517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f623fa1-0eee-4d7b-a458-5c5e9ee7d8cc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:37.239802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:37.240912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:37.241201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:37.241477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:37.243271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:37.244794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:37.246810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 148/290] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:36 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:37.253173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:37.281340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15cd9c-4531-4829-bdb6-02052a662840 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:37.332358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:37.362513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:37.363132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:37.363406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:37.363674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:37.375175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6b25c-304c-4740-be24-5766a1b15be7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:37.376191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:37.412919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:37.413613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:37.413897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:37.414177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/cd7402dc049c40a1a1a85b7ffad5f979/roles/5615ab8f7bfd40e5a904031e70bcc62b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:37.416116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d69e08be64483acd3b1c8fd75c744, user_id=cd7402dc049c40a1a1a85b7ffad5f979, role_id=5615ab8f7bfd40e5a904031e70bcc62b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:37.422674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:37.446939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 149/291] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:42:37 2022] PUT /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/users/cd7402dc049c40a1a1a85b7ffad5f979/roles/5615ab8f7bfd40e5a904031e70bcc62b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:37.494757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:37.496581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:37.626074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 143/292] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:42:37.632173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:37.773235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25b2f-f157-481a-9842-176cf05e7f79 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:37.797277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:37.809073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc94122-9b37-4c32-bf6f-7554ac6006c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:37.810301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc94122-9b37-4c32-bf6f-7554ac6006c9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:37.810636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc94122-9b37-4c32-bf6f-7554ac6006c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:37.810915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc94122-9b37-4c32-bf6f-7554ac6006c9 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:37.812171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc94122-9b37-4c32-bf6f-7554ac6006c9 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:37.813668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc94122-9b37-4c32-bf6f-7554ac6006c9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:37.815646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 150/293] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:37 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:37.825349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:37.893202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:37.982846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d90678-0864-4258-9d0b-50d1cb96fe5a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:38.032954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f97a7-92ac-4740-b644-f2d6def54e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:38.033872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f97a7-92ac-4740-b644-f2d6def54e80 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:38.034332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f97a7-92ac-4740-b644-f2d6def54e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:38.034808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f97a7-92ac-4740-b644-f2d6def54e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:38.035771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f97a7-92ac-4740-b644-f2d6def54e80 None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=154812ddd98748e4b36746123ff5e64f) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:38.036269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f97a7-92ac-4740-b644-f2d6def54e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:38.037873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f97a7-92ac-4740-b644-f2d6def54e80 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:38.045166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 144/294] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:37 2022] GET /identity/v3/role_assignments?user.id=154812ddd98748e4b36746123ff5e64f&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=61a194f6aba541de8c32f53dfc572624 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:38.425681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bf133-ebc8-4d13-87e3-487dc7c4b3dc None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:38.426186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bf133-ebc8-4d13-87e3-487dc7c4b3dc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:38.426283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bf133-ebc8-4d13-87e3-487dc7c4b3dc None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:38.427965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 151/295] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:38.438244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d1ee1-3608-42a1-b5ec-88e4962064b7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:38.438244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d1ee1-3608-42a1-b5ec-88e4962064b7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:38.438244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d1ee1-3608-42a1-b5ec-88e4962064b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:38.441038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edef5a-b286-42d8-a5ba-e332a721362f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:38.441331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edef5a-b286-42d8-a5ba-e332a721362f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:38.441606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edef5a-b286-42d8-a5ba-e332a721362f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:38.443085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 152/296] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:38.450829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcba007-db25-496b-90f3-f2cab80484b2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:38.451148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcba007-db25-496b-90f3-f2cab80484b2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:38.451425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcba007-db25-496b-90f3-f2cab80484b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:38.458612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d88d1ee1-3608-42a1-b5ec-88e4962064b7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:38.460104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d88d1ee1-3608-42a1-b5ec-88e4962064b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:38.480424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bfcba007-db25-496b-90f3-f2cab80484b2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:38.480836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bfcba007-db25-496b-90f3-f2cab80484b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:38.572007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 153/297] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:38.577548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 145/298] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:38.744330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50091-3967-4dae-bbf6-f889b4989334 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:38.744330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50091-3967-4dae-bbf6-f889b4989334 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:38.744330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e50091-3967-4dae-bbf6-f889b4989334 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:38.746597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 154/299] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:38.752030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f7e18-9e76-4169-8267-20c32aa3ad96 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:38.752593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f7e18-9e76-4169-8267-20c32aa3ad96 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:38.753084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f7e18-9e76-4169-8267-20c32aa3ad96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:38.771631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0f0f7e18-9e76-4169-8267-20c32aa3ad96 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:38.773656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0f0f7e18-9e76-4169-8267-20c32aa3ad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:38.800977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:38.847420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 146/300] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:38.869936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e50091-3967-4dae-bbf6-f889b4989334 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:38.943084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f7e18-9e76-4169-8267-20c32aa3ad96 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:38.978943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e50091-3967-4dae-bbf6-f889b4989334 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.001313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:39.018839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.019472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.019751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.020023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.034008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:39.041984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:39.042834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:39.043612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:39.044197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 155/301] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:39.051329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.051448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f7e18-9e76-4169-8267-20c32aa3ad96 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.087397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.088030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.088333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.088655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.089756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.091198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.092849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 147/302] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:39.098443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.128026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:39.157605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47efcc7-816b-47b0-b4b9-7eb404c88823 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.193253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.193869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.194146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.194675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.195610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:39.195912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.197313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.201214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78759b-470f-4a62-a507-ab22de2fc3b3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.205779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 156/303] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:39.209703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.237058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.237664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.237941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.238210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] PATH_INFO: `/v3/groups` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.238827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.240268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.264457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 148/304] 10.222.0.50 () {60 vars in 1134 bytes} [Tue Aug 30 01:42:39 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:42:39.265997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.266738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:39.345224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05beea-0f2f-4988-bd3d-d6275906abdc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.395991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:39.398677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.400011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.401016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.401903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.409027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62575-477e-4ec2-9490-9cb72c3e8eaf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.421743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:39.427926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:39.429771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:39.430619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 157/305] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:39.436924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.457217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.457844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.458121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.458393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] PATH_INFO: `/v3/users/14c01ae5f2fd4c44bd3388476d8a3b9e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.459596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] RBAC: Authorizing `identity:get_user(user_id=14c01ae5f2fd4c44bd3388476d8a3b9e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.461130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.463502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 149/306] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:39 2022] GET /identity/v3/users/14c01ae5f2fd4c44bd3388476d8a3b9e => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:39.468358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.546597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a107f0e-99ba-467b-995c-21c82e3e6854 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.549293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:39.590319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.590932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.591198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c700d-4de0-460a-b43e-3c4845e477cc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.591303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.591472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.592137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:39.592468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.594127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.605425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 158/307] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:39.610800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.631986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e053ed-2a31-455d-9d8d-113a48d76fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.633735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e053ed-2a31-455d-9d8d-113a48d76fba None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.634246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e053ed-2a31-455d-9d8d-113a48d76fba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.635100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e053ed-2a31-455d-9d8d-113a48d76fba None admin] PATH_INFO: `/v3/system/users/14c01ae5f2fd4c44bd3388476d8a3b9e/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.637381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e053ed-2a31-455d-9d8d-113a48d76fba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=14c01ae5f2fd4c44bd3388476d8a3b9e, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.639114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e053ed-2a31-455d-9d8d-113a48d76fba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.664357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 150/308] 10.222.0.50 () {58 vars in 1334 bytes} [Tue Aug 30 01:42:39 2022] PUT /identity/v3/system/users/14c01ae5f2fd4c44bd3388476d8a3b9e/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:42:39.733085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cd0db-9fb2-438f-9941-2d6aa3f45f34 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.776931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.778020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.778526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.779018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] PATH_INFO: `/v3/users/dc52b45e7dfb491a80e6dfe891caef85` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.787582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced59b-befe-4de3-9445-10c50646dae4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.787889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced59b-befe-4de3-9445-10c50646dae4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.788140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced59b-befe-4de3-9445-10c50646dae4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.790422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 151/309] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:39.796112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] RBAC: Authorizing `identity:get_user(user_id=dc52b45e7dfb491a80e6dfe891caef85)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.796558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419854fd-dff0-430a-bdf8-0f6d980f025c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.797097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419854fd-dff0-430a-bdf8-0f6d980f025c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.797572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419854fd-dff0-430a-bdf8-0f6d980f025c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.797769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.799856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 159/310] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:39 2022] GET /identity/v3/users/dc52b45e7dfb491a80e6dfe891caef85 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:39.806886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:39.825040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-419854fd-dff0-430a-bdf8-0f6d980f025c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:39.826434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-419854fd-dff0-430a-bdf8-0f6d980f025c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:39.833665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:39.846503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:39.917045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8acda7-ab13-4485-8899-fb259487d9fb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:39.921619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 152/311] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:39.957111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:39.957726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:39.958002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:39.958327 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:39.959119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] RBAC: Adding query filter params (role.id=5615ab8f7bfd40e5a904031e70bcc62b, user.id=dc52b45e7dfb491a80e6dfe891caef85, scope.system=all) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:39.959427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:39.960974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:39.966698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 160/312] 10.222.0.50 () {56 vars in 1308 bytes} [Tue Aug 30 01:42:39 2022] GET /identity/v3/role_assignments?user.id=dc52b45e7dfb491a80e6dfe891caef85&scope.system=all&role.id=5615ab8f7bfd40e5a904031e70bcc62b => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:40.196828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419854fd-dff0-430a-bdf8-0f6d980f025c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:40.305186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419854fd-dff0-430a-bdf8-0f6d980f025c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:40.340347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:40.341337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:40.341807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:40.342266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:40.353605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:40.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:40.359584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:40.360408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:40.363429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 153/313] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:40.369122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:40.463475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:40.481564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf31a-7e10-4fa7-9579-6093de20991f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:40.518584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:40.519185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:40.519461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:40.519737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:40.520426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:40.520730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:40.522247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:40.531056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 161/314] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:40.536813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:40.553884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:40.643349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63ce57-c36e-4bd7-84c3-c18a6eff00e7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:40.678258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:40.678888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:40.679191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:40.679464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] PATH_INFO: `/v3/users/cd7402dc049c40a1a1a85b7ffad5f979` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:40.680639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=cd7402dc049c40a1a1a85b7ffad5f979)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:40.682079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:40.683825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 154/315] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:40 2022] GET /identity/v3/users/cd7402dc049c40a1a1a85b7ffad5f979 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:40.689527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:40.793002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcaa2e-8810-4f9f-9fb7-9ffafb803f9b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:40.828919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:40.829584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:40.829868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:40.830146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:40.848679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:40.902900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:40.904382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:41.006563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 162/316] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:41.012482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:41.102638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:41.123757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce11bd-a2b0-4bb0-842e-5ae1d2ce6fd4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:41.158924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353d0d5-75a5-4700-adf7-9eb8962d0459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:41.159558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d0d5-75a5-4700-adf7-9eb8962d0459 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.159840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d0d5-75a5-4700-adf7-9eb8962d0459 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.160318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d0d5-75a5-4700-adf7-9eb8962d0459 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.161456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353d0d5-75a5-4700-adf7-9eb8962d0459 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:41.162970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353d0d5-75a5-4700-adf7-9eb8962d0459 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:41.164977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 155/317] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:41 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:41.170743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:41.219959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:41.277388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208ac99-3c8c-40dd-8982-34052a25a3ec None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:41.312997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d567f-7f3e-4c64-935c-06f2240dc494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:41.313610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d567f-7f3e-4c64-935c-06f2240dc494 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.313887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d567f-7f3e-4c64-935c-06f2240dc494 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.314157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d567f-7f3e-4c64-935c-06f2240dc494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.314891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d567f-7f3e-4c64-935c-06f2240dc494 None admin] RBAC: Adding query filter params (role.id=5615ab8f7bfd40e5a904031e70bcc62b, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744, user.id=cd7402dc049c40a1a1a85b7ffad5f979) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:41.315198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d567f-7f3e-4c64-935c-06f2240dc494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:41.316671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d567f-7f3e-4c64-935c-06f2240dc494 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:41.323120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 163/318] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:42:41 2022] GET /identity/v3/role_assignments?user.id=cd7402dc049c40a1a1a85b7ffad5f979&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=5615ab8f7bfd40e5a904031e70bcc62b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:41.426904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b869ef-f8ec-4826-ae79-d9ec10472dad None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.427412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b869ef-f8ec-4826-ae79-d9ec10472dad None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.427574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b869ef-f8ec-4826-ae79-d9ec10472dad None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.429217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 156/319] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:41.436120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d90ce-0011-49be-8ab7-5ae6b64ca088 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.436579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d90ce-0011-49be-8ab7-5ae6b64ca088 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.436953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d90ce-0011-49be-8ab7-5ae6b64ca088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.456701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bd6d90ce-0011-49be-8ab7-5ae6b64ca088 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:41.457905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd6d90ce-0011-49be-8ab7-5ae6b64ca088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:41.527235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 164/320] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:41.764044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234c4bf-90fb-4083-a764-59562a8fd60b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.764430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234c4bf-90fb-4083-a764-59562a8fd60b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.764625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234c4bf-90fb-4083-a764-59562a8fd60b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.766089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 157/321] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:41.770964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e189bc-d901-4014-a838-fcc7cc11e6b8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.771255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e189bc-d901-4014-a838-fcc7cc11e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.771527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e189bc-d901-4014-a838-fcc7cc11e6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.790779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-30e189bc-d901-4014-a838-fcc7cc11e6b8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:41.791939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-30e189bc-d901-4014-a838-fcc7cc11e6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:41.814047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234c4bf-90fb-4083-a764-59562a8fd60b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:41.830555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:41.865231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 165/322] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:41.873819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253194d7-445c-4414-b862-bbd5156fe3f3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.874208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253194d7-445c-4414-b862-bbd5156fe3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.874483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253194d7-445c-4414-b862-bbd5156fe3f3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.875886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 166/323] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:41.880542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a1dd6-0731-4e1e-96f3-672d687c82e0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.880857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a1dd6-0731-4e1e-96f3-672d687c82e0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.881079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a1dd6-0731-4e1e-96f3-672d687c82e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.900051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2f7a1dd6-0731-4e1e-96f3-672d687c82e0 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:41.901267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2f7a1dd6-0731-4e1e-96f3-672d687c82e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:41.929320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234c4bf-90fb-4083-a764-59562a8fd60b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:41.964841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 167/324] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:41.965666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:41.966543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:41.966965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:41.967446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:41.969494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:41.972448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:41.975615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 158/325] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:41.981612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a1dd6-0731-4e1e-96f3-672d687c82e0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.079522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a1dd6-0731-4e1e-96f3-672d687c82e0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.081093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:42.115835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.116969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.117435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.117891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] PATH_INFO: `/v3/groups` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.119020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:42.121617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:42.134768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.140912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 168/326] 10.222.0.50 () {60 vars in 1134 bytes} [Tue Aug 30 01:42:41 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:42:42.155687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:42.249612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c7448-772a-454e-af0e-1c0a6209a272 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.258643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.296102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.297148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.297897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.297897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.312577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:42.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:42.320576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:42.321391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 159/327] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:42.328925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.389982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca578-4ca2-45cf-9c9d-41be21a2cb09 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.417938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:42.437752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.438553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.438835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.439432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.445999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4c173-d366-42c2-ba2b-23fcbbb672b7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.455557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:42.463435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:42.464554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:42.465325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 169/328] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:42.470537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.494417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.495026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.495304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.495574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.496719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:42.497021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:42.498831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:42.498942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:42.510129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 160/329] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:42.515400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.604660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531a6cd-0ad0-4b60-944f-97e4af26f496 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.640564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.641098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.641375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.641646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.642306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:42.642609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:42.643995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:42.644291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691c05a-bdc5-452b-b49b-6fd315944d54 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.653419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 170/330] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:42.659243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.696718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.697365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.697645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.697918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] PATH_INFO: `/v3/users/14c01ae5f2fd4c44bd3388476d8a3b9e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.700376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] RBAC: Authorizing `identity:get_user(user_id=14c01ae5f2fd4c44bd3388476d8a3b9e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:42.701845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:42.704417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 161/331] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:42 2022] GET /identity/v3/users/14c01ae5f2fd4c44bd3388476d8a3b9e => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:42.712450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.724647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:42.775829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:42.797281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee9f1d-1b41-44a3-a36e-1da80ef7cd2a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.838838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa78480-be5d-4d05-9919-e6b713eda4fb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:42.841358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.842868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.843144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.843413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] PATH_INFO: `/v3/users/dc52b45e7dfb491a80e6dfe891caef85` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.844598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] RBAC: Authorizing `identity:get_user(user_id=dc52b45e7dfb491a80e6dfe891caef85)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:42.846049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:42.848308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 171/332] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:42 2022] GET /identity/v3/users/dc52b45e7dfb491a80e6dfe891caef85 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:42.854870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:42.878647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b30466-9ba6-4db2-8d19-67975c2de433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:42.879258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30466-9ba6-4db2-8d19-67975c2de433 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:42.879539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30466-9ba6-4db2-8d19-67975c2de433 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:42.879857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30466-9ba6-4db2-8d19-67975c2de433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:42.880715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30466-9ba6-4db2-8d19-67975c2de433 None admin] RBAC: Adding query filter params (role.id=61a194f6aba541de8c32f53dfc572624, user.id=14c01ae5f2fd4c44bd3388476d8a3b9e, scope.system=all) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:42.881029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30466-9ba6-4db2-8d19-67975c2de433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:42.883139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30466-9ba6-4db2-8d19-67975c2de433 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:42.896547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 162/333] 10.222.0.50 () {56 vars in 1308 bytes} [Tue Aug 30 01:42:42 2022] GET /identity/v3/role_assignments?user.id=14c01ae5f2fd4c44bd3388476d8a3b9e&scope.system=all&role.id=61a194f6aba541de8c32f53dfc572624 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:42.975892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797dee9-d964-4926-900a-ab44cb134d4e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:43.013021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9d4c8-e233-4af1-8e93-75bf404486fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:43.013694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d4c8-e233-4af1-8e93-75bf404486fe None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:43.013968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d4c8-e233-4af1-8e93-75bf404486fe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:43.014241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d4c8-e233-4af1-8e93-75bf404486fe None admin] PATH_INFO: `/v3/system/users/dc52b45e7dfb491a80e6dfe891caef85/roles/5615ab8f7bfd40e5a904031e70bcc62b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:43.015713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9d4c8-e233-4af1-8e93-75bf404486fe None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dc52b45e7dfb491a80e6dfe891caef85, role_id=5615ab8f7bfd40e5a904031e70bcc62b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:43.017158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9d4c8-e233-4af1-8e93-75bf404486fe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:43.028263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 172/334] 10.222.0.50 () {58 vars in 1334 bytes} [Tue Aug 30 01:42:42 2022] PUT /identity/v3/system/users/dc52b45e7dfb491a80e6dfe891caef85/roles/5615ab8f7bfd40e5a904031e70bcc62b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:44.529314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c4688-0e00-48a5-835c-bdee84a312f0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:44.529908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c4688-0e00-48a5-835c-bdee84a312f0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:44.529908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c4688-0e00-48a5-835c-bdee84a312f0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:44.531430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 163/335] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:44.536815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45a8c-25e8-44ea-b136-3e10994343f5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:44.537241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45a8c-25e8-44ea-b136-3e10994343f5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:44.537446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45a8c-25e8-44ea-b136-3e10994343f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:44.546969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998a323-568f-490f-b19f-654ef7c4b8c4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:44.547220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998a323-568f-490f-b19f-654ef7c4b8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:44.547455 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998a323-568f-490f-b19f-654ef7c4b8c4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:44.548939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 164/336] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:44.555457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164be1-ad6b-45ff-b7c5-baaa176aad32 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:44.555737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164be1-ad6b-45ff-b7c5-baaa176aad32 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:44.555998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90164be1-ad6b-45ff-b7c5-baaa176aad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:44.575007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d3e45a8c-25e8-44ea-b136-3e10994343f5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:44.576205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d3e45a8c-25e8-44ea-b136-3e10994343f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:44.584275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-90164be1-ad6b-45ff-b7c5-baaa176aad32 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:44.585534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90164be1-ad6b-45ff-b7c5-baaa176aad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:44.599848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:44.714793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:44.731525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 173/337] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:44.734940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 165/338] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:45.238861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d053e9-9a89-4bcb-bb70-b5c7aea12968 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.239258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d053e9-9a89-4bcb-bb70-b5c7aea12968 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.239448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d053e9-9a89-4bcb-bb70-b5c7aea12968 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.241081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 174/339] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:45.247139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c76a8-208a-4766-82c2-941a8c56993a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.247429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c76a8-208a-4766-82c2-941a8c56993a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.247705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c76a8-208a-4766-82c2-941a8c56993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.261386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d053e9-9a89-4bcb-bb70-b5c7aea12968 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:45.285752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-879c76a8-208a-4766-82c2-941a8c56993a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:45.286945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-879c76a8-208a-4766-82c2-941a8c56993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:45.411056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 166/340] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:45.411776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c76a8-208a-4766-82c2-941a8c56993a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:45.450679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d053e9-9a89-4bcb-bb70-b5c7aea12968 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:45.524663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721b037-66f6-4d16-bf3b-3a3b150ae5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:45.525622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b037-66f6-4d16-bf3b-3a3b150ae5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.526127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b037-66f6-4d16-bf3b-3a3b150ae5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.526595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b037-66f6-4d16-bf3b-3a3b150ae5f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.532374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c76a8-208a-4766-82c2-941a8c56993a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:45.551111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721b037-66f6-4d16-bf3b-3a3b150ae5f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:45.555642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c721b037-66f6-4d16-bf3b-3a3b150ae5f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:45.557639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:45.558531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:45.559618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 175/341] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:45.560692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6f0f-b988-401c-a54b-1d3e3f2a8e72 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.560949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6f0f-b988-401c-a54b-1d3e3f2a8e72 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.561229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6f0f-b988-401c-a54b-1d3e3f2a8e72 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.562938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 176/342] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:45.568025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b452e-fd49-4708-80f2-584c9d3f1289 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.568403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b452e-fd49-4708-80f2-584c9d3f1289 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.568625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b452e-fd49-4708-80f2-584c9d3f1289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.583548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:45.584197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.584507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.584778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.588320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:45.598998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6a1b452e-fd49-4708-80f2-584c9d3f1289 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:45.600530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6a1b452e-fd49-4708-80f2-584c9d3f1289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:45.612291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:45.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:45.618790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:45.620123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 167/343] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:45.622450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:45.729741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 177/344] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:45.730695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b452e-fd49-4708-80f2-584c9d3f1289 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:45.761935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740f6cb-1fd9-4221-b1a5-6d5588557a2d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:45.812621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:45.813717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.814197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.814727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.815556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:45.816017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:45.817789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:45.827528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 168/345] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:45.833302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:45.903914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b452e-fd49-4708-80f2-584c9d3f1289 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:45.917727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:45.947883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:45.951090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:45.951380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:45.951651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:45.952350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:45.952653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:45.954080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:45.963469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 178/346] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:45.964114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:45.969045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:46.064023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e7b77-125d-4653-8f49-ad418d0a5c06 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:46.127096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c4906-73f7-4e3d-9ce0-5aa3aa830cc5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:46.153504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:46.154124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.154400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.154669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.169109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:46.169716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.170001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.170275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.235045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:46.237130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:46.238653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:46.242215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:46.243719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:46.290026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:46.362016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 169/347] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:46.362927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:46.374458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 179/348] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:46.375726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:46.500033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:46.509181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa789-54b6-4451-a6dc-fcb22d509358 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:46.516892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139565d-745e-4faa-92a6-23b00b74e6d6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:46.553323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18d890-9cb6-4b00-a9f3-98546c01b785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:46.553323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18d890-9cb6-4b00-a9f3-98546c01b785 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.553323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18d890-9cb6-4b00-a9f3-98546c01b785 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.553943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18d890-9cb6-4b00-a9f3-98546c01b785 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.559573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:46.560214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.560532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.560940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.566544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18d890-9cb6-4b00-a9f3-98546c01b785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8d18d890-9cb6-4b00-a9f3-98546c01b785 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:46.572424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:46.573435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 01:42:46.574216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:46.575686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:46.577591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 180/349] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:46.578888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ae940-7bd0-40e6-b3d1-4b0ad657a672 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.579180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ae940-7bd0-40e6-b3d1-4b0ad657a672 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.579457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ae940-7bd0-40e6-b3d1-4b0ad657a672 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:46.581043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:46.581891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:46.582668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:46.583259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 181/350] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:46.583259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ae940-7bd0-40e6-b3d1-4b0ad657a672 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:46.583384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 170/351] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:46.583384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:46.596541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:46.707628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9d6fa-2def-450c-be24-67a625786993 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:46.714151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ae940-7bd0-40e6-b3d1-4b0ad657a672 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:46.755832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:46.756502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.756775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.757045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.758204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:46.759638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:46.760815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5935a-2d68-456a-9e6f-5174dc33f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:46.761585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 182/352] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:46 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:46.762157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:46.762412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5935a-2d68-456a-9e6f-5174dc33f102 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.762951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5935a-2d68-456a-9e6f-5174dc33f102 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.763501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5935a-2d68-456a-9e6f-5174dc33f102 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.764924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5935a-2d68-456a-9e6f-5174dc33f102 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:46.766566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5935a-2d68-456a-9e6f-5174dc33f102 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:46.768927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 171/353] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:46 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:46.771430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef5ec6-59e2-441e-b2d6-6dff0e09dc3b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.771922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef5ec6-59e2-441e-b2d6-6dff0e09dc3b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.772481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef5ec6-59e2-441e-b2d6-6dff0e09dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.792705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ddef5ec6-59e2-441e-b2d6-6dff0e09dc3b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:46.796184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ddef5ec6-59e2-441e-b2d6-6dff0e09dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:46.800121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:46.887678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 172/354] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:46.888569 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef5ec6-59e2-441e-b2d6-6dff0e09dc3b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:46.894384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:46.907530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6086b25-8d98-4d20-9a67-232f25abb451 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:46.952274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:46.952896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:46.953177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:46.953458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:46.954108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:46.954415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:46.955780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:46.965629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 183/355] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:46.966365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.023401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef5ec6-59e2-441e-b2d6-6dff0e09dc3b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.066248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.067308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.067782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.068321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.069171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:47.069813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.071261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.081390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 173/356] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:47.082204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.084209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:47.104331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636106dc-b05d-4429-b383-b8a2e65c219b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.152638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1992fa5-879b-46b7-851b-65ba39c15d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.153274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1992fa5-879b-46b7-851b-65ba39c15d17 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.153617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1992fa5-879b-46b7-851b-65ba39c15d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.153822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1992fa5-879b-46b7-851b-65ba39c15d17 None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.162346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:47.164886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1992fa5-879b-46b7-851b-65ba39c15d17 None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.166180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1992fa5-879b-46b7-851b-65ba39c15d17 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.168141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 184/357] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:46 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:47.170629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aff92c-de62-4222-98ed-e1dc147f0129 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.170629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aff92c-de62-4222-98ed-e1dc147f0129 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.170629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aff92c-de62-4222-98ed-e1dc147f0129 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.171134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 185/358] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:47.171823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aff92c-de62-4222-98ed-e1dc147f0129 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.206174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4bea6-faf5-40db-9153-12fad1720453 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.246538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.247175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.247452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.247722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.249343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.250771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.258154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 174/359] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:47.258305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.291415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aff92c-de62-4222-98ed-e1dc147f0129 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.334220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.335152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.336236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.337361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] PATH_INFO: `/v3/users/dc52b45e7dfb491a80e6dfe891caef85` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.337773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] RBAC: Authorizing `identity:get_user(user_id=dc52b45e7dfb491a80e6dfe891caef85)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.339839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.341894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 186/360] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/users/dc52b45e7dfb491a80e6dfe891caef85 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:47.344361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.369817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:47.376935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d948be-9e1e-426e-98ec-c5ccf1356b50 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.421305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20931b-b529-42e3-a530-d2f8fe1fd965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.421922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20931b-b529-42e3-a530-d2f8fe1fd965 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.422200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20931b-b529-42e3-a530-d2f8fe1fd965 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.422472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20931b-b529-42e3-a530-d2f8fe1fd965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.425406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:47.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2faf12-fcc7-4fc4-8cec-227abf0c1c59 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.503879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20931b-b529-42e3-a530-d2f8fe1fd965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.505788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20931b-b529-42e3-a530-d2f8fe1fd965 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.517473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.518122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.518400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.518673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.531011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:47.536789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:47.537672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 187/361] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:47.538834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.627451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 175/362] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:47.628956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2b7d-1710-406c-805e-aebfee85826a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.629330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2b7d-1710-406c-805e-aebfee85826a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.629707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2b7d-1710-406c-805e-aebfee85826a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.634481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:47.649224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2cba2b7d-1710-406c-805e-aebfee85826a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:47.650414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2cba2b7d-1710-406c-805e-aebfee85826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:47.655867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eac37-2267-459a-a1e1-09abc637ed82 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.684045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:47.693551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.694160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.694440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.697066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.697066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=8b71340f7ded406b9f92818f9e00a227) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:47.697239 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.698637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.705637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 188/363] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=fdf7d3a219d340db99038e400383416b => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:47.705637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.737921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 176/364] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:47.738805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2b7d-1710-406c-805e-aebfee85826a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.859411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098b7ec-553f-4d83-a7c5-db5fbf7ed54b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.866183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2b7d-1710-406c-805e-aebfee85826a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:47.903136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.903755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.904036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.904354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.905087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=8b71340f7ded406b9f92818f9e00a227) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:47.905394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.906831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.907621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:47.908632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:47.909104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:47.910036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:47.911236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] RBAC: Adding query filter params (role.id=5615ab8f7bfd40e5a904031e70bcc62b, user.id=dc52b45e7dfb491a80e6dfe891caef85, scope.system=all) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:47.911739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:47.912654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 189/365] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=61a194f6aba541de8c32f53dfc572624 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:47.913502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.913914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:47.940663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:47.948707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 177/366] 10.222.0.50 () {56 vars in 1308 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/role_assignments?user.id=dc52b45e7dfb491a80e6dfe891caef85&scope.system=all&role.id=5615ab8f7bfd40e5a904031e70bcc62b => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:47.949196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:47.995184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:48.074155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e37461-257f-4484-8e5e-902eccc74c88 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.115977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.117734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.118776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.119424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.120440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:48.120968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:48.122618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:48.133296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 190/367] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:48.137577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.162341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87885c4-03d7-4624-a266-1e27f15430d4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.222455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:48.251404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.252051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.252354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.252635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.256856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:48.258811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:48.260650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 178/368] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:47 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:48.261282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.274717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81afc-2907-4ba8-8ee8-b15a67d078a8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.339225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.339868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.340142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.341440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.367165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:48.372628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:48.373498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:48.374278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 191/369] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:48.374911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.407472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:48.418423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53623e18-b0a7-4400-b206-de13f9235bba None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.455594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.456648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.457121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.457608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.484057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb80b3-fe7b-4e10-8934-d52745d00bc0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.515499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:48.521867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.522581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.522894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.523172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.524941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:48.526504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:48.528171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 192/370] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:48.528891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.539172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:48.541804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:48.639093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3826ee-1927-48b9-87ef-d5df77b674a8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.640840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:48.653710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 179/371] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:48.654851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.674666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.675357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.675550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.675832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.676782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:48.676885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:48.678195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:48.687604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 193/372] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:48.688543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.753165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:48.773771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339326ee-d471-49e2-9aa1-6de2a8d5cd9c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.796740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae77ffb-94a4-41e7-9d09-917706ece178 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.812150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.813207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.813700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.814158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.815073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:48.815664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:48.817382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:48.823663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 180/373] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=61a194f6aba541de8c32f53dfc572624 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:48.824792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.834097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.835102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.835102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.835336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:48.836520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:48.837973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:48.839643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 194/374] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:48.849326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:48.897287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:48.951899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd7717-fa7a-421b-969f-4447cc29de39 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.976525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b8089-dfff-4a77-9e4c-249fa4d735b0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:48.991495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:48.992904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:48.993466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:48.993876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:49.010079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:49.025369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:49.026115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:49.026497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:49.026876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:49.028119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:49.029710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:49.031694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 195/375] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:49.037974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:49.071234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:49.072945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:49.136765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:49.153750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30affba3-9682-4f55-9d17-0e378c1fde94 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:49.180021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 181/376] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:49.185973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:49.197692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:49.198322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:49.198597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:49.198873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:49.199616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:49.199990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:49.202043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:49.209092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 196/377] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:49 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=fdf7d3a219d340db99038e400383416b => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:49.305683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e8244-fd53-485f-a13e-5abda6cc2ce1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:49.344095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:49.344776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:49.345057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:49.345328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] PATH_INFO: `/v3/projects/e7c73461b5344be681351a70b1beafa9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:49.346668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] RBAC: Authorizing `identity:get_project(project_id=e7c73461b5344be681351a70b1beafa9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:49.348116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:49.350212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 182/378] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:49 2022] GET /identity/v3/projects/e7c73461b5344be681351a70b1beafa9 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:49.356027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:49.444187 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:49.492684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df94f-fa13-433d-b5b8-0082b510e5a6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:49.527926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd71cf-6f21-4661-8be0-5270645e5224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:49.528643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd71cf-6f21-4661-8be0-5270645e5224 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:49.528924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd71cf-6f21-4661-8be0-5270645e5224 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:49.529195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd71cf-6f21-4661-8be0-5270645e5224 None admin] PATH_INFO: `/v3/groups/006e7f3041714e3c9913cf1f2d6b77ac` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:49.534228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd71cf-6f21-4661-8be0-5270645e5224 None admin] RBAC: Authorizing `identity:get_group(group_id=006e7f3041714e3c9913cf1f2d6b77ac)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:49.535676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd71cf-6f21-4661-8be0-5270645e5224 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:49.537602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 197/379] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:49 2022] GET /identity/v3/groups/006e7f3041714e3c9913cf1f2d6b77ac => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:49.543672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:49.579620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:49.669664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc874-efb0-4ad1-b1c3-71ab3f3c8c44 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:49.706477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0aa1e-511f-46bf-950e-44a61522d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:49.707009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0aa1e-511f-46bf-950e-44a61522d882 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:49.707289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0aa1e-511f-46bf-950e-44a61522d882 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:49.707560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0aa1e-511f-46bf-950e-44a61522d882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:49.708303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0aa1e-511f-46bf-950e-44a61522d882 None admin] RBAC: Adding query filter params (group.id=006e7f3041714e3c9913cf1f2d6b77ac, role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=e7c73461b5344be681351a70b1beafa9) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:49.708612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0aa1e-511f-46bf-950e-44a61522d882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:49.710024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0aa1e-511f-46bf-950e-44a61522d882 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:49.717081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 183/380] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:49 2022] GET /identity/v3/role_assignments?group.id=006e7f3041714e3c9913cf1f2d6b77ac&scope.project.id=e7c73461b5344be681351a70b1beafa9&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:50.457361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc2b2-b796-465f-9469-2faed1da59bc None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:50.457899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc2b2-b796-465f-9469-2faed1da59bc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:50.457899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc2b2-b796-465f-9469-2faed1da59bc None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:50.459495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 198/381] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:50.474455 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03751823-f9a0-42a2-93d5-3a783131dd6b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:50.474743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03751823-f9a0-42a2-93d5-3a783131dd6b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:50.475025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03751823-f9a0-42a2-93d5-3a783131dd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:50.505216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-03751823-f9a0-42a2-93d5-3a783131dd6b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:50.506420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-03751823-f9a0-42a2-93d5-3a783131dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:50.593967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 184/382] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:50.602535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0c770-19aa-48ee-832e-79887f3a6573 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:50.603094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0c770-19aa-48ee-832e-79887f3a6573 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:50.603569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0c770-19aa-48ee-832e-79887f3a6573 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:50.606343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 199/383] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:50.610721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e9193-6134-46c2-b127-efce001ccf61 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:50.611477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e9193-6134-46c2-b127-efce001ccf61 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:50.611969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e9193-6134-46c2-b127-efce001ccf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:50.638002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-555e9193-6134-46c2-b127-efce001ccf61 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:50.639505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-555e9193-6134-46c2-b127-efce001ccf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:50.650773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:50.717577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 185/384] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:50.891602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0c770-19aa-48ee-832e-79887f3a6573 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.020946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0c770-19aa-48ee-832e-79887f3a6573 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.045656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:51.063615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.064266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.064550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.064819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.077903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e9193-6134-46c2-b127-efce001ccf61 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.084265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:51.087983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:51.088946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:51.089960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 200/385] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:51.094692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.250802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0954e2-f66c-4d24-b393-a2563d3d6ce6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.290871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.291790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.292216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.292853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.293646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:51.294078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:51.295997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:51.306008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 201/386] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:51.314417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.316975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e9193-6134-46c2-b127-efce001ccf61 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.392713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:51.394434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f026d-2d60-4e2b-9e7f-96560e0bf87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.394434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f026d-2d60-4e2b-9e7f-96560e0bf87f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.394654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f026d-2d60-4e2b-9e7f-96560e0bf87f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.394920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f026d-2d60-4e2b-9e7f-96560e0bf87f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.406003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f026d-2d60-4e2b-9e7f-96560e0bf87f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-598f026d-2d60-4e2b-9e7f-96560e0bf87f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:51.411260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:51.412123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:51.412926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 186/387] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:51.414495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cecb3-19e2-4670-a29e-43138f12d8a8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.414802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cecb3-19e2-4670-a29e-43138f12d8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.415088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cecb3-19e2-4670-a29e-43138f12d8a8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.420420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 187/388] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:51.421437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cecb3-19e2-4670-a29e-43138f12d8a8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.432587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:51.433896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373baa04-d340-4c3d-893f-55c638688957 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.476164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32de89-dfec-4f31-8f22-8fac1118208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.478052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32de89-dfec-4f31-8f22-8fac1118208c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.478714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32de89-dfec-4f31-8f22-8fac1118208c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.479148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32de89-dfec-4f31-8f22-8fac1118208c None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.480880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32de89-dfec-4f31-8f22-8fac1118208c None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:51.482891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32de89-dfec-4f31-8f22-8fac1118208c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:51.485566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 202/389] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:51 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:51.487622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87953892-f16c-4bcd-9bd3-2ba557e84a13 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.488060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87953892-f16c-4bcd-9bd3-2ba557e84a13 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.488502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87953892-f16c-4bcd-9bd3-2ba557e84a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.511447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-87953892-f16c-4bcd-9bd3-2ba557e84a13 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:51.513179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-87953892-f16c-4bcd-9bd3-2ba557e84a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:51.525165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cecb3-19e2-4670-a29e-43138f12d8a8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.561406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e1e7b-bd3b-4654-9b9b-01cc20ed1988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.562012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e1e7b-bd3b-4654-9b9b-01cc20ed1988 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.562286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e1e7b-bd3b-4654-9b9b-01cc20ed1988 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.562555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e1e7b-bd3b-4654-9b9b-01cc20ed1988 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.563207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e1e7b-bd3b-4654-9b9b-01cc20ed1988 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:51.563547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e1e7b-bd3b-4654-9b9b-01cc20ed1988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:51.564869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e1e7b-bd3b-4654-9b9b-01cc20ed1988 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:51.578192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 188/390] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:51.578192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52889d-4881-41b2-b483-3eba51471b5c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.578192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52889d-4881-41b2-b483-3eba51471b5c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.578192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52889d-4881-41b2-b483-3eba51471b5c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.578192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 189/391] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:51.578192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52889d-4881-41b2-b483-3eba51471b5c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.590273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 203/392] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:51.591380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87953892-f16c-4bcd-9bd3-2ba557e84a13 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.643351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:51.669725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:51.701073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87953892-f16c-4bcd-9bd3-2ba557e84a13 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.708599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52889d-4881-41b2-b483-3eba51471b5c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.737475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873db25-79ed-4949-a796-b22e5dba3ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.738408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873db25-79ed-4949-a796-b22e5dba3ffa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.738887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873db25-79ed-4949-a796-b22e5dba3ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.739410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873db25-79ed-4949-a796-b22e5dba3ffa None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.740850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873db25-79ed-4949-a796-b22e5dba3ffa None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:51.743024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873db25-79ed-4949-a796-b22e5dba3ffa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:51.744817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 204/393] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:51 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:51.745597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.746280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.746611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.746916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.747391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5ace-d503-4d63-881a-041dd745770d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.747862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5ace-d503-4d63-881a-041dd745770d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.748158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:51.748518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5ace-d503-4d63-881a-041dd745770d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.749708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:51.751646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 190/394] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:51 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:51.752499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.769428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a65a5ace-d503-4d63-881a-041dd745770d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:51.770943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a65a5ace-d503-4d63-881a-041dd745770d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:51.843792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 205/395] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:51.845275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a5ace-d503-4d63-881a-041dd745770d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.857602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b5e9d-f157-4eb8-8cde-02fc7a1bc126 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.892374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:51.894822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.895460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.895739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.896131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:51.897850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:51.900073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:51.902653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 191/396] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:51 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:51.913120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:51.917375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:51.960945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a5ace-d503-4d63-881a-041dd745770d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:51.997564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:51.998517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:51.998987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:51.999470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227/groups/54a34567912a445c85703db86758b2ef/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.001847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] RBAC: Authorizing `identity:create_grant(project_id=8b71340f7ded406b9f92818f9e00a227, group_id=54a34567912a445c85703db86758b2ef, role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:52.003445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:52.026041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 206/397] 10.222.0.50 () {58 vars in 1442 bytes} [Tue Aug 30 01:42:51 2022] PUT /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227/groups/54a34567912a445c85703db86758b2ef/roles/fdf7d3a219d340db99038e400383416b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:52.032964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.045463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3f3c2-93e7-43c0-8a13-0890a8adc4cb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.091934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083065b5-228d-40c1-9a87-cdf0a8751be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.093054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083065b5-228d-40c1-9a87-cdf0a8751be8 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.093345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083065b5-228d-40c1-9a87-cdf0a8751be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.093628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083065b5-228d-40c1-9a87-cdf0a8751be8 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227/groups/54a34567912a445c85703db86758b2ef/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.095961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083065b5-228d-40c1-9a87-cdf0a8751be8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b71340f7ded406b9f92818f9e00a227, group_id=54a34567912a445c85703db86758b2ef, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:52.097607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083065b5-228d-40c1-9a87-cdf0a8751be8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:52.126612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 192/398] 10.222.0.50 () {58 vars in 1442 bytes} [Tue Aug 30 01:42:51 2022] PUT /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227/groups/54a34567912a445c85703db86758b2ef/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:52.134506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3b5f5-70a0-41e0-95c3-c6db5fcd582f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.134833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3b5f5-70a0-41e0-95c3-c6db5fcd582f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.135068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3b5f5-70a0-41e0-95c3-c6db5fcd582f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.141061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 193/399] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:42:52.142645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cb3a6-2746-44cd-8f5d-109e4087b164 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.142932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cb3a6-2746-44cd-8f5d-109e4087b164 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.143214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cb3a6-2746-44cd-8f5d-109e4087b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.164081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bd6cb3a6-2746-44cd-8f5d-109e4087b164 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:52.165483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bd6cb3a6-2746-44cd-8f5d-109e4087b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:52.203167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8036f-7dae-45d3-ab3a-9d22dafe932e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.207550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:52.250321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:52.256785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.257797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.258106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.258339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.262502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 194/400] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:52.263186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cb3a6-2746-44cd-8f5d-109e4087b164 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.285892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:52.293602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:52.295929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:52.297060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 207/401] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:52.299178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.426903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cb3a6-2746-44cd-8f5d-109e4087b164 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.464671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.465288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.465567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.466183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.478750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:52.488793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafafa29-2dcc-45f8-93e4-a65317912b8f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:52.490698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:52.491604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:52.492810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 195/402] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:52.499482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.539935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.540499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.540776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.541047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.541776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:52.542084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:52.543584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:52.552671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 208/403] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:52.558762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.601558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:52.611522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:52.668734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ac904-590a-4014-91d3-38c6d6512213 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.701245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1711d4b-0a72-44c6-a21d-bfcb8cb5c781 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.715134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.716085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.716583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.717044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.717894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:52.718385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:52.719948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:52.728990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 196/404] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:52.730058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.765369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.766008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.766288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.766565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.767685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:52.769164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:52.775895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 209/405] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:52.776780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.869004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edde95-e5dd-48c3-9373-d61346db9b79 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.886433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:52.888045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:52.900796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0920ac-f25a-4ebe-b381-056f947c1328 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:52.910066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.910821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.911389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.911508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.922122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:52.927933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:52.928848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 197/406] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:52.935333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:52.950306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:52.950956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:52.951215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:52.951488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:52.952637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:52.954090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:52.960212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 210/407] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:52.960212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:53.065949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab59fe1-842d-4b89-8f8e-9a0a07251e9d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.077193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b343259-1057-40de-b4fd-81f1e26e31b2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.108638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.110181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.110772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.111558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.112559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:53.113206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.114329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.115033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.115306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.115426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.115718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.117490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.119262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.121055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 198/408] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:53.126453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:53.131335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 211/409] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:53.133963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:53.161180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:53.184337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:53.240233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376d418-f372-4b77-b9c3-5a699c7dbd40 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.245622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a832c0-a503-4822-8936-4953173ec8df None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.279117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.280116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.280593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.281152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/groups/54a34567912a445c85703db86758b2ef/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.284270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d69e08be64483acd3b1c8fd75c744, group_id=54a34567912a445c85703db86758b2ef, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.284564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.285314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.285662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.286004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.287196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.289398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.289905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.292423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 199/410] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:53 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:53.292423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:53.311845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 212/411] 10.222.0.50 () {58 vars in 1442 bytes} [Tue Aug 30 01:42:53 2022] PUT /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/groups/54a34567912a445c85703db86758b2ef/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:53.313012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:53.408977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458de83b-214c-4f35-b6df-f105849f8eff None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.412335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:53.424509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:53.435063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f086a-27b9-423d-867b-a8c8a6d95e75 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.447914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.448848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.449200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.449613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] PATH_INFO: `/v3/groups/006e7f3041714e3c9913cf1f2d6b77ac` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.450784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] RBAC: Authorizing `identity:get_group(group_id=006e7f3041714e3c9913cf1f2d6b77ac)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.452216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.454142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 200/412] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:53 2022] GET /identity/v3/groups/006e7f3041714e3c9913cf1f2d6b77ac => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:53.459801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:53.501663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.502386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.502636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.502914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/groups/54a34567912a445c85703db86758b2ef/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.510213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] RBAC: Authorizing `identity:create_grant(project_id=fb1d69e08be64483acd3b1c8fd75c744, group_id=54a34567912a445c85703db86758b2ef, role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.511689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.558493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 213/413] 10.222.0.50 () {58 vars in 1442 bytes} [Tue Aug 30 01:42:53 2022] PUT /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744/groups/54a34567912a445c85703db86758b2ef/roles/fdf7d3a219d340db99038e400383416b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:42:53.567944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718033b-306d-44ce-b017-eb10291fd9e9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.606641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6ab3-4ccf-4a27-aad9-92a226d04ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.607332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6ab3-4ccf-4a27-aad9-92a226d04ebe None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.607615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6ab3-4ccf-4a27-aad9-92a226d04ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.607887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6ab3-4ccf-4a27-aad9-92a226d04ebe None admin] PATH_INFO: `/v3/projects/e7c73461b5344be681351a70b1beafa9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.609085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6ab3-4ccf-4a27-aad9-92a226d04ebe None admin] RBAC: Authorizing `identity:get_project(project_id=e7c73461b5344be681351a70b1beafa9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.610756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6ab3-4ccf-4a27-aad9-92a226d04ebe None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.613259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 201/414] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:53 2022] GET /identity/v3/projects/e7c73461b5344be681351a70b1beafa9 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:53.619528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:53.761864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97e7b5-4c48-4ece-a979-abc514b244af None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:53.797338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee886d3-6510-41f5-adac-24dd06ac2a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:53.798458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee886d3-6510-41f5-adac-24dd06ac2a0a None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:53.798934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee886d3-6510-41f5-adac-24dd06ac2a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:53.799405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee886d3-6510-41f5-adac-24dd06ac2a0a None admin] PATH_INFO: `/v3/projects/e7c73461b5344be681351a70b1beafa9/groups/006e7f3041714e3c9913cf1f2d6b77ac/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:53.801538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee886d3-6510-41f5-adac-24dd06ac2a0a None admin] RBAC: Authorizing `identity:create_grant(project_id=e7c73461b5344be681351a70b1beafa9, group_id=006e7f3041714e3c9913cf1f2d6b77ac, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:53.803196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee886d3-6510-41f5-adac-24dd06ac2a0a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:53.828166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 214/415] 10.222.0.50 () {58 vars in 1442 bytes} [Tue Aug 30 01:42:53 2022] PUT /identity/v3/projects/e7c73461b5344be681351a70b1beafa9/groups/006e7f3041714e3c9913cf1f2d6b77ac/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:42:54.268220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83721b-b7af-4ee1-8d25-da4df29c88f6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:54.268764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83721b-b7af-4ee1-8d25-da4df29c88f6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:54.268764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83721b-b7af-4ee1-8d25-da4df29c88f6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:54.270377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 202/416] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:54.281730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f99ebf-63d0-478d-8dd9-2dec213eef3d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:54.282014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f99ebf-63d0-478d-8dd9-2dec213eef3d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:54.282345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f99ebf-63d0-478d-8dd9-2dec213eef3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:54.290576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:54.308957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-04f99ebf-63d0-478d-8dd9-2dec213eef3d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:54.310146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-04f99ebf-63d0-478d-8dd9-2dec213eef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:54.406862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 215/417] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:54.454551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5731914-7243-4416-ad9a-59ef28e9ee44 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:54.454845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5731914-7243-4416-ad9a-59ef28e9ee44 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:54.455107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5731914-7243-4416-ad9a-59ef28e9ee44 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:54.456657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 203/418] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:54.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b80fc-2de1-4d35-8935-da281bc7be76 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:54.462384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b80fc-2de1-4d35-8935-da281bc7be76 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:54.462663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b80fc-2de1-4d35-8935-da281bc7be76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:54.481406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dc3b80fc-2de1-4d35-8935-da281bc7be76 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:54.482576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dc3b80fc-2de1-4d35-8935-da281bc7be76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:54.550634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 216/419] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:54.789985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5731914-7243-4416-ad9a-59ef28e9ee44 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:54.848073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:54.871723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b80fc-2de1-4d35-8935-da281bc7be76 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:54.953874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5731914-7243-4416-ad9a-59ef28e9ee44 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:54.976630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:55.001817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b80fc-2de1-4d35-8935-da281bc7be76 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:55.044628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:55.045256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.045531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.045798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.049885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:55.050502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.050772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.051037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.062896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:55.068137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:55.069020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:55.070344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:55.071215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 204/420] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:55.079395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:55.080228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:55.085717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:55.086773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:55.088517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:55.091121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 217/421] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:42:55.095890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:55.219054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75753-e796-4cf3-b3de-19f7ff1a4a5b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:55.228579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:55.255256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:55.260333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.260333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.260333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.260333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:55.260333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:55.260333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:55.268992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 205/422] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:55.274081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:55.282326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161a7c-2536-4351-9fa5-6a1c1194703c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:55.312617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:55.322702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:55.323446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.323800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.324323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.325743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:55.326241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:55.328053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:55.337774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 218/423] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:55.342820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:55.516210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33604d-33e8-439c-a569-815a04f8d1c4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:55.557869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:55.558633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.558924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.559202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.586285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d85101-ee39-4728-8160-2374c32a44ad None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:55.602729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:55.634599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:55.636077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:55.720983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ebbac-b174-43f2-9724-853f180d48ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:55.721614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebbac-b174-43f2-9724-853f180d48ea None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.721900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebbac-b174-43f2-9724-853f180d48ea None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.722174 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebbac-b174-43f2-9724-853f180d48ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.742205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 206/424] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:55.747522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:55.830633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ebbac-b174-43f2-9724-853f180d48ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:55.831217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:55.832964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ebbac-b174-43f2-9724-853f180d48ea None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:55.871760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:55.905710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b79474-fff1-434c-8de2-cb09bcb2e3da None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:55.942241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f9f18-4783-4755-b1a5-addc11c8d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:55.942869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f9f18-4783-4755-b1a5-addc11c8d15d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.943157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f9f18-4783-4755-b1a5-addc11c8d15d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.943433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f9f18-4783-4755-b1a5-addc11c8d15d None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.944624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f9f18-4783-4755-b1a5-addc11c8d15d None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:55.946055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f9f18-4783-4755-b1a5-addc11c8d15d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:55.947777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 207/425] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:55 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:55.948960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255627a-da15-415e-ac3d-7d80cd6682a7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.949242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255627a-da15-415e-ac3d-7d80cd6682a7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.949532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255627a-da15-415e-ac3d-7d80cd6682a7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.950885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 208/426] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:55.951715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d5d9-4b51-4eeb-8df9-06affd1fea25 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.952031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d5d9-4b51-4eeb-8df9-06affd1fea25 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.952280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d5d9-4b51-4eeb-8df9-06affd1fea25 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.953760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 209/427] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:55.958355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d5d9-4b51-4eeb-8df9-06affd1fea25 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:55.970099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 219/428] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:55.972296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec228c41-6035-4734-94b7-d2db6aa0bd68 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:55.972762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec228c41-6035-4734-94b7-d2db6aa0bd68 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:55.973338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec228c41-6035-4734-94b7-d2db6aa0bd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:55.993747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ec228c41-6035-4734-94b7-d2db6aa0bd68 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:55.995272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ec228c41-6035-4734-94b7-d2db6aa0bd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:56.063354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 220/429] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:56.064615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935185-d651-4bc8-a163-f752e1415a91 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.064902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935185-d651-4bc8-a163-f752e1415a91 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.065220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935185-d651-4bc8-a163-f752e1415a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.066529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d5d9-4b51-4eeb-8df9-06affd1fea25 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.083095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-08935185-d651-4bc8-a163-f752e1415a91 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:56.084205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-08935185-d651-4bc8-a163-f752e1415a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:56.094139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:56.108292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.108928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.109206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.109480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.110764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:56.112701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:56.114799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 210/430] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:55 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:56.115476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.128828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:56.162879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 221/431] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:42:56.164768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935185-d651-4bc8-a163-f752e1415a91 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.224647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf46da5-d5ba-43cd-a8a8-3261d47850d3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.262639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.263268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.263544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.264071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] PATH_INFO: `/v3/projects/8b71340f7ded406b9f92818f9e00a227` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.265800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] RBAC: Authorizing `identity:get_project(project_id=8b71340f7ded406b9f92818f9e00a227)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:56.267249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:56.269046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 211/432] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/projects/8b71340f7ded406b9f92818f9e00a227 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:56.277284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.285614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935185-d651-4bc8-a163-f752e1415a91 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.329159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cd89b-2e22-4a5f-86e1-0f8ef53a28a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.330108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cd89b-2e22-4a5f-86e1-0f8ef53a28a2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.330688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cd89b-2e22-4a5f-86e1-0f8ef53a28a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cd89b-2e22-4a5f-86e1-0f8ef53a28a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.332220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cd89b-2e22-4a5f-86e1-0f8ef53a28a2 None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=8b71340f7ded406b9f92818f9e00a227) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:56.332771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cd89b-2e22-4a5f-86e1-0f8ef53a28a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:56.334600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cd89b-2e22-4a5f-86e1-0f8ef53a28a2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:56.342645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 222/433] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=61a194f6aba541de8c32f53dfc572624 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:56.344705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf600fd-63e2-48dc-9ea8-790105ea27a0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.345249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf600fd-63e2-48dc-9ea8-790105ea27a0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.345725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf600fd-63e2-48dc-9ea8-790105ea27a0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.352147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 223/434] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:42:56.354475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4842b1-00cb-4855-b77c-f11c5cb8f035 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.354987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4842b1-00cb-4855-b77c-f11c5cb8f035 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.355485 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4842b1-00cb-4855-b77c-f11c5cb8f035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.379150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1a4842b1-00cb-4855-b77c-f11c5cb8f035 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:42:56.380750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a4842b1-00cb-4855-b77c-f11c5cb8f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:42:56.384203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:56.421744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:56.472401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a29d2-dc36-419d-8724-99d844c0c7b6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.476624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 224/435] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:42:56.481610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4842b1-00cb-4855-b77c-f11c5cb8f035 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.523345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.524116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.524457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.524702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.525929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:56.527490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:56.529470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 212/436] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:56.530253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.613350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4842b1-00cb-4855-b77c-f11c5cb8f035 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.647041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95287d-3213-4af7-80ab-9d825d7ba9a9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.651909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.652581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.652863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.653141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.660634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:56.667997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:56.672976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:56.673841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:42:56.674627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 225/437] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:56.675335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.696280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.697219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.697501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.697777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.711210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:56.716626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:56.717518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 213/438] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:56.718981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.721862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:56.803543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85491991-667c-40d6-b28a-361656f33e77 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.845178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.845384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0b3f0-9179-42d6-acc4-67e2a16e014f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.845823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.846121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.846399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.847124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=8b71340f7ded406b9f92818f9e00a227) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:56.847430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:56.848964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:56.854847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 226/439] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=8b71340f7ded406b9f92818f9e00a227&role.id=fdf7d3a219d340db99038e400383416b => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:56.855435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.900657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:56.901210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:56.901492 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:56.902324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:56.903032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:56.903790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:56.905558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:56.922947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 214/440] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:56.922947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:56.923535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:56.969694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d09282-9483-4080-bfd7-72505b84045e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:56.993627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.010958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.011903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.012426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.012700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.013365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:57.013665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.015032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.023998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 227/441] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:57.024671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.045684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffed2f-35ad-4c22-a5a0-49484703e3eb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.086858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.086858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.086858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.087197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.098074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:42:57.103995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:42:57.105183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 215/442] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:42:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:42:57.106167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.146972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd194e-1a75-442c-ad91-30692e9268b9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.178855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.193768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.193768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.193768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.193768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.226732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a273f2-e898-45cf-9fe8-618564d75a49 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.250408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.261997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.262650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.262873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.263146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.263297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.264798 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.329225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.330719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.360464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 228/443] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:57.361544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.385181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.430673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 216/444] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:57.431429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.456115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.456851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e91a6-c77e-4ec2-8ad6-63851417e56d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.491039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.491662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.491936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.492205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.492889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:57.493186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.494544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.502230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 229/445] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:57.502230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.553048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e7e98-0c99-48db-b9f3-077fed0dc0bd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.583128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.588972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.589600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.589892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.590169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.591288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.592761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.594362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 217/446] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:57.595110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.598258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30d07d-0248-4446-a489-a5af6cc02e8a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.630949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.631603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.631841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.632137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] PATH_INFO: `/v3/projects/fb1d69e08be64483acd3b1c8fd75c744` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.633276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] RBAC: Authorizing `identity:get_project(project_id=fb1d69e08be64483acd3b1c8fd75c744)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.634701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.636330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 230/447] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/projects/fb1d69e08be64483acd3b1c8fd75c744 => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:57.637089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.684171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.693675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8da8bd-68ed-4f47-a0eb-3a6b68e262a4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.728042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.728685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.728959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.729232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.736869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec42a78-9cec-4f30-b1d9-30c9e1b06b2d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.769496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.770103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.770378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.770645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.771733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.773716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.775256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 231/448] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:57.776007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.794905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.796349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.804442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.881411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749496f1-a7b2-4252-992a-24eee414057c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:57.886538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:57.896387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 218/449] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:42:57.897191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.914892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:57.915514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:57.915788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:57.916058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] PATH_INFO: `/v3/groups/54a34567912a445c85703db86758b2ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:57.917171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] RBAC: Authorizing `identity:get_group(group_id=54a34567912a445c85703db86758b2ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:57.918587 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:57.920115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 232/450] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/groups/54a34567912a445c85703db86758b2ef => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:57.920890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:57.994839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eb596-2f1c-4212-8b4b-111280b328db None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:58.009906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:58.019584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a185d52-b113-4c4a-b9ba-e82c196299cf None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:58.028055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:58.028675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:58.028941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:58.029211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:58.029922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:58.030224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:58.031623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:58.037240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 219/451] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=fdf7d3a219d340db99038e400383416b => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:58.037973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:58.058813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:58.059453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:58.059733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:58.060037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] PATH_INFO: `/v3/projects/e7c73461b5344be681351a70b1beafa9` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:58.061750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] RBAC: Authorizing `identity:get_project(project_id=e7c73461b5344be681351a70b1beafa9)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:58.062764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:58.064703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 233/452] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:42:57 2022] GET /identity/v3/projects/e7c73461b5344be681351a70b1beafa9 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:58.070888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:58.103406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:58.153323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d55e7-a010-4513-80db-3fc115d206c8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:58.192980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:58.193634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:58.195285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:58.195285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:58.195445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea0a9a-dc68-44ba-a5c6-4f85031c7b37 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:58.195517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] RBAC: Adding query filter params (group.id=54a34567912a445c85703db86758b2ef, role.id=61a194f6aba541de8c32f53dfc572624, scope.project.id=fb1d69e08be64483acd3b1c8fd75c744) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:58.195804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:58.197285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:58.203563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 220/453] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:58 2022] GET /identity/v3/role_assignments?group.id=54a34567912a445c85703db86758b2ef&scope.project.id=fb1d69e08be64483acd3b1c8fd75c744&role.id=61a194f6aba541de8c32f53dfc572624 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:42:58.239662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23563e74-3abb-4144-a12e-8f1c300555c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:58.240401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23563e74-3abb-4144-a12e-8f1c300555c9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:58.240678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23563e74-3abb-4144-a12e-8f1c300555c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:58.240946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23563e74-3abb-4144-a12e-8f1c300555c9 None admin] PATH_INFO: `/v3/groups/006e7f3041714e3c9913cf1f2d6b77ac` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:58.242635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23563e74-3abb-4144-a12e-8f1c300555c9 None admin] RBAC: Authorizing `identity:get_group(group_id=006e7f3041714e3c9913cf1f2d6b77ac)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:58.244114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23563e74-3abb-4144-a12e-8f1c300555c9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:58.245810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 234/454] 10.222.0.50 () {56 vars in 1180 bytes} [Tue Aug 30 01:42:58 2022] GET /identity/v3/groups/006e7f3041714e3c9913cf1f2d6b77ac => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:42:58.251949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:42:58.359530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d38d1-de4f-461b-8b4c-be867cc922df None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:42:58.380178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:42:58.394456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227265dc-381e-488e-ac1f-b31081ebd492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:42:58.395053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227265dc-381e-488e-ac1f-b31081ebd492 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:42:58.395329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227265dc-381e-488e-ac1f-b31081ebd492 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:42:58.395596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227265dc-381e-488e-ac1f-b31081ebd492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:42:58.396342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227265dc-381e-488e-ac1f-b31081ebd492 None admin] RBAC: Adding query filter params (group.id=006e7f3041714e3c9913cf1f2d6b77ac, role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=e7c73461b5344be681351a70b1beafa9) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:42:58.396645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227265dc-381e-488e-ac1f-b31081ebd492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:42:58.398091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227265dc-381e-488e-ac1f-b31081ebd492 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:42:58.403890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 221/455] 10.222.0.50 () {56 vars in 1376 bytes} [Tue Aug 30 01:42:58 2022] GET /identity/v3/role_assignments?group.id=006e7f3041714e3c9913cf1f2d6b77ac&scope.project.id=e7c73461b5344be681351a70b1beafa9&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:00.765680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735969d3-9308-4dc7-96e2-fe1b17c04bd7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:00.766212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735969d3-9308-4dc7-96e2-fe1b17c04bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:00.766212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735969d3-9308-4dc7-96e2-fe1b17c04bd7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:00.767838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 235/456] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:00.773591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23f29-6276-4767-b784-1783a0ee6685 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:00.773998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23f29-6276-4767-b784-1783a0ee6685 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:00.774185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23f29-6276-4767-b784-1783a0ee6685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:00.797566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e3b23f29-6276-4767-b784-1783a0ee6685 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:00.798482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e3b23f29-6276-4767-b784-1783a0ee6685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:00.898776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 222/457] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:01.126173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0d2a8-ffc3-4876-8997-7b4bde0b3d4d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.126506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0d2a8-ffc3-4876-8997-7b4bde0b3d4d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.126627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0d2a8-ffc3-4876-8997-7b4bde0b3d4d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.128114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 236/458] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:01.136729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dcb9d-b1fa-413b-9f27-89a55e6eda93 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.137020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dcb9d-b1fa-413b-9f27-89a55e6eda93 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.137299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dcb9d-b1fa-413b-9f27-89a55e6eda93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.163902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b37dcb9d-b1fa-413b-9f27-89a55e6eda93 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:01.165138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b37dcb9d-b1fa-413b-9f27-89a55e6eda93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:01.253525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 223/459] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:01.255069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da8b0-d7bc-45ee-91b6-051df1752f89 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.255355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da8b0-d7bc-45ee-91b6-051df1752f89 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.255628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da8b0-d7bc-45ee-91b6-051df1752f89 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.257742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 237/460] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:01.271941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f9898-02ae-43ff-9980-6ee3ca0c0a84 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.272224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f9898-02ae-43ff-9980-6ee3ca0c0a84 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.272590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f9898-02ae-43ff-9980-6ee3ca0c0a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.308539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-774f9898-02ae-43ff-9980-6ee3ca0c0a84 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:01.309740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-774f9898-02ae-43ff-9980-6ee3ca0c0a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:01.315170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da8b0-d7bc-45ee-91b6-051df1752f89 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:01.347850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:01.384719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:01.418973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 224/461] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:01.493830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da8b0-d7bc-45ee-91b6-051df1752f89 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:01.537989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:01.539258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.539532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.539799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.554860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:01.556335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:01.558576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f9898-02ae-43ff-9980-6ee3ca0c0a84 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:01.572921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:01.573934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:01.574785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:01.574785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:01.574785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:01.577312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 238/462] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:01.582203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:01.674163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f9898-02ae-43ff-9980-6ee3ca0c0a84 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:01.703181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:01.712889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb03f54-4f60-4581-a3ce-ac3bc52cf31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:01.713522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb03f54-4f60-4581-a3ce-ac3bc52cf31d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.713802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb03f54-4f60-4581-a3ce-ac3bc52cf31d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.714070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb03f54-4f60-4581-a3ce-ac3bc52cf31d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.723968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb03f54-4f60-4581-a3ce-ac3bc52cf31d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:01.725440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb03f54-4f60-4581-a3ce-ac3bc52cf31d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:01.728079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb2fab-b267-439b-91bb-7fa4f8ea764a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-dfb03f54-4f60-4581-a3ce-ac3bc52cf31d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:01.739880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:01.740963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 225/463] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d29b24-6f57-4f71-943e-d7f276e9682a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d29b24-6f57-4f71-943e-d7f276e9682a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d29b24-6f57-4f71-943e-d7f276e9682a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 226/464] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:01.746593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d29b24-6f57-4f71-943e-d7f276e9682a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:01.773716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:01.775562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:01.776314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.776685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.777085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.777722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:01.778598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:01.780152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:01.803225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 239/465] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:01.803225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:01.859585 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d29b24-6f57-4f71-943e-d7f276e9682a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:01.894984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce309933-ced0-42b1-bf7b-31cf0526cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:01.895613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce309933-ced0-42b1-bf7b-31cf0526cc96 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.895894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce309933-ced0-42b1-bf7b-31cf0526cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.896173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce309933-ced0-42b1-bf7b-31cf0526cc96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.905293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eecbb7-56f4-474d-8885-fd3718468cab None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:01.907985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce309933-ced0-42b1-bf7b-31cf0526cc96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:01.909473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce309933-ced0-42b1-bf7b-31cf0526cc96 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ce309933-ced0-42b1-bf7b-31cf0526cc96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:01.923813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:01.924795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:01.925634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:01.925634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:01.925634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:01.925634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 227/466] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:01.930602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dbeab-2fd7-40c0-9c37-261fbf6355cd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.930887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dbeab-2fd7-40c0-9c37-261fbf6355cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.931304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dbeab-2fd7-40c0-9c37-261fbf6355cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.943532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:01.944141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:01.945009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:01.945287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:01.945925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:01.946226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:01.947601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:01.952457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b67dbeab-2fd7-40c0-9c37-261fbf6355cd None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:01.953631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b67dbeab-2fd7-40c0-9c37-261fbf6355cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:01.954048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:01.959201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 240/467] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:01.960352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.032601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 228/468] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:02.033452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dbeab-2fd7-40c0-9c37-261fbf6355cd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.044199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:02.091040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52118caf-a462-41d1-856d-e9de63cafa6f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:02.127251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:02.127869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.128439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.128722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.129367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:02.131763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:02.159196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 241/469] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:43:01 2022] POST /identity/v3/users => generated 246 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:02.160386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.181415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dbeab-2fd7-40c0-9c37-261fbf6355cd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:02.223012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:02.224007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.224511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.225009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.225912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:02.226404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:02.227962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:02.241898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 229/470] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:02.253562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.273279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:02.321773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c54042-8339-4b1f-9772-46819282d1dc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:02.359245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:02.359871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.360155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.360479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.361468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:02.362955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:02.364724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:02.391193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 242/471] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:43:02 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:02.400801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.424821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733f0ba-d265-4c2d-a36b-550dd3f0d527 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:02.472114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287defe-53ea-4037-91fc-35a072bfdc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:02.472722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287defe-53ea-4037-91fc-35a072bfdc07 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.473000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287defe-53ea-4037-91fc-35a072bfdc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.473275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287defe-53ea-4037-91fc-35a072bfdc07 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.473988 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287defe-53ea-4037-91fc-35a072bfdc07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:02.475433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287defe-53ea-4037-91fc-35a072bfdc07 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:02.507037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 230/472] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:43:02 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:02.509120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443955b-0170-47ae-9d92-c2efde26aa97 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.509414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443955b-0170-47ae-9d92-c2efde26aa97 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.509694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443955b-0170-47ae-9d92-c2efde26aa97 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.511165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 231/473] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:02.517079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c1180-01c6-416b-8468-453215e3ab5d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.517370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c1180-01c6-416b-8468-453215e3ab5d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.517643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c1180-01c6-416b-8468-453215e3ab5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.535249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ab3da-4f11-4a6d-a1c1-d8309f8a89f0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:02.556558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:02.573351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-932c1180-01c6-416b-8468-453215e3ab5d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:02.574560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-932c1180-01c6-416b-8468-453215e3ab5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:02.584862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:02.585565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.585889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.586166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.602479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:02.604927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:02.627681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:02.628667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:02.629489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:02.629489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:02.629489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:02.629489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 243/474] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:02.634665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.682890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 232/475] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:43:02.750891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f55d4-5e1d-40cf-bf26-508365c28151 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:02.786084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:02.787429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:02.787880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:02.788963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:02.789774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:02.790266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:02.791822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:02.805189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 244/476] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:02.810474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c1180-01c6-416b-8468-453215e3ab5d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.889165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:02.952860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c1180-01c6-416b-8468-453215e3ab5d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:02.961284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:02.998922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:03.005097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:03.007440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:03.007718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:03.007995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:03.008666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:03.013412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:03.057428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 233/477] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:43:02 2022] POST /identity/v3/users => generated 249 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:43:03.108465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9cabc-3b4a-444b-ab03-0393b8705835 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:03.150361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:03.150993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:03.151267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:03.151536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:03.152234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:03.156298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:03.176298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.sql.core [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 01:43:03.176298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 01:43:03.176298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [parameters: {'id': '01944db9bdfc4369a86379786e390692', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 01:43:03.176298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81429) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application result.read() Aug 30 01:43:03.197558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:43:03.198473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application yield resource Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise self.value Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application session.commit() Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 01:43:03.199769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self.session.flush() Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application compat.raise_( Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 01:43:03.200622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:43:03.201511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application result.read() Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:43:03.203476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [parameters: {'id': '01944db9bdfc4369a86379786e390692', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:03.205398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 01:43:03.207476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:03.209405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 245/478] 10.222.0.50 () {60 vars in 1131 bytes} [Tue Aug 30 01:43:02 2022] POST /identity/v3/roles => generated 144 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:03.209553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:03.362446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8ab8b-7390-448c-b790-72bdb1293142 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:03.375033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:03.400939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee42f5-bc17-46d8-b2cf-c877e01b9962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:03.401571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee42f5-bc17-46d8-b2cf-c877e01b9962 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:03.401848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee42f5-bc17-46d8-b2cf-c877e01b9962 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:03.402120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee42f5-bc17-46d8-b2cf-c877e01b9962 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:03.412205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee42f5-bc17-46d8-b2cf-c877e01b9962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b4ee42f5-bc17-46d8-b2cf-c877e01b9962 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:03.417539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:03.419549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:03.420897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 234/479] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:03.424404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:03.526715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:03.542868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5de6a-7384-4c8b-94cb-f754ee98a51c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:03.579856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d61d8-fa3e-440b-95d2-dec3c2598a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:03.580757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d61d8-fa3e-440b-95d2-dec3c2598a02 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:03.581206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d61d8-fa3e-440b-95d2-dec3c2598a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:03.581654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d61d8-fa3e-440b-95d2-dec3c2598a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:03.582502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d61d8-fa3e-440b-95d2-dec3c2598a02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:03.582986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d61d8-fa3e-440b-95d2-dec3c2598a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:03.584531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d61d8-fa3e-440b-95d2-dec3c2598a02 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:03.593146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 246/480] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:04.573902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f759c-3884-4dec-afe0-4eae6a8343e1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:04.574901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f759c-3884-4dec-afe0-4eae6a8343e1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:04.575377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f759c-3884-4dec-afe0-4eae6a8343e1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:04.577258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 235/481] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:04.582905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83302a59-e655-4c24-84b6-12ea3087da5d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:04.583311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83302a59-e655-4c24-84b6-12ea3087da5d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:04.583492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83302a59-e655-4c24-84b6-12ea3087da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:04.606832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-83302a59-e655-4c24-84b6-12ea3087da5d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:04.608081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-83302a59-e655-4c24-84b6-12ea3087da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:04.700101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 247/482] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:04.891292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4088-edb6-47b0-afda-47fe1365bc41 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:04.891777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4088-edb6-47b0-afda-47fe1365bc41 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:04.892202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4088-edb6-47b0-afda-47fe1365bc41 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:04.893931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 236/483] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:04.899459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2ab5-7e79-4e4f-8fed-bb85fcc38cd0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:04.899743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2ab5-7e79-4e4f-8fed-bb85fcc38cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:04.900012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2ab5-7e79-4e4f-8fed-bb85fcc38cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:04.925266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-817f2ab5-7e79-4e4f-8fed-bb85fcc38cd0 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:04.926435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-817f2ab5-7e79-4e4f-8fed-bb85fcc38cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:05.014940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 248/484] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:05.056647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8de89-7a02-4217-9cad-bb7b04e26b88 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.057100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8de89-7a02-4217-9cad-bb7b04e26b88 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.057505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8de89-7a02-4217-9cad-bb7b04e26b88 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.059904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 237/485] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:05.064972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c12cb-75bf-42ed-9bf2-b1c939284c5c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.065253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c12cb-75bf-42ed-9bf2-b1c939284c5c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.065527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c12cb-75bf-42ed-9bf2-b1c939284c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.086532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c80c12cb-75bf-42ed-9bf2-b1c939284c5c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:05.087953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c80c12cb-75bf-42ed-9bf2-b1c939284c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:05.090429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:05.110841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8de89-7a02-4217-9cad-bb7b04e26b88 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.174882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 249/486] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:05.251866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8de89-7a02-4217-9cad-bb7b04e26b88 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:05.301052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:05.301696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.301976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.302249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.318497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:05.323926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:05.324836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:05.325624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 238/487] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:05.330821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c12cb-75bf-42ed-9bf2-b1c939284c5c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.427853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.478796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c12cb-75bf-42ed-9bf2-b1c939284c5c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:05.494161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:05.520270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:05.520270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.520270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.520270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.520662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:05.520662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:05.520662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:05.524426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:05.528182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 250/488] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:05.534589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.619324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34b3d-c4b4-4751-a485-2140a69e75a7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:05.662579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e0325-39b6-4340-a6d9-2e5927d01d76 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:05.667398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:05.668030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.668346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.668623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.686221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:05.691765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:05.692685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:05.693497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 239/489] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:05.694139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.705452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:05.706450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.706932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.707444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.719023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:05.720809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:05.736785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:05.737717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:05.738542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:05.738542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:05.738542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:05.748268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 251/490] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:05.748268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.800059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:05.814671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc0d4f-5674-4c17-8e2f-5538cc9149c5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:05.814836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:05.852804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d62cf-87c6-4fe8-b474-bfdd0691bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:05.853433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d62cf-87c6-4fe8-b474-bfdd0691bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.853709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d62cf-87c6-4fe8-b474-bfdd0691bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.853977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d62cf-87c6-4fe8-b474-bfdd0691bdc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.864483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d62cf-87c6-4fe8-b474-bfdd0691bdc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:05.869643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba1657-84b2-43ea-8d74-123f58019239 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a22d62cf-87c6-4fe8-b474-bfdd0691bdc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:05.869774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:05.870583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:05.871349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 240/491] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:05.876497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006af67-2a28-437a-9104-aa6fc9a43000 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.876830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006af67-2a28-437a-9104-aa6fc9a43000 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.877110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006af67-2a28-437a-9104-aa6fc9a43000 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.878488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 241/492] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:05.879021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006af67-2a28-437a-9104-aa6fc9a43000 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.912546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:05.914082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:05.914577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:05.915043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:05.916010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:05.916631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:05.918081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:05.931003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 252/493] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:05.931003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:05.992021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006af67-2a28-437a-9104-aa6fc9a43000 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.027661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3254ed-b630-41a3-b911-4a4e63ecb42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.028585 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3254ed-b630-41a3-b911-4a4e63ecb42b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.029043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3254ed-b630-41a3-b911-4a4e63ecb42b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.029311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3254ed-b630-41a3-b911-4a4e63ecb42b None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.029942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3254ed-b630-41a3-b911-4a4e63ecb42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:06.030235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3254ed-b630-41a3-b911-4a4e63ecb42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.031595 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3254ed-b630-41a3-b911-4a4e63ecb42b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.037318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413edd91-ed8f-4723-a70e-bc8bbb24ce08 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.045411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 242/494] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:06.047190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c68a-bca3-470c-a3a4-b25a6fd8cb05 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.047766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c68a-bca3-470c-a3a4-b25a6fd8cb05 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.048544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c68a-bca3-470c-a3a4-b25a6fd8cb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.051860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.070872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0322c68a-bca3-470c-a3a4-b25a6fd8cb05 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:06.072203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0322c68a-bca3-470c-a3a4-b25a6fd8cb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:06.074887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.077900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.078577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.078899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.079116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.079775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:06.080519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.081893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.096234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 253/495] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:06.096234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.145673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 243/496] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:06.146418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322c68a-bca3-470c-a3a4-b25a6fd8cb05 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.208328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9e93-a3c2-4cd2-a377-064df667774f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.243519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.244060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.244375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.244657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.256264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.258279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.258383 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322c68a-bca3-470c-a3a4-b25a6fd8cb05 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:06.274308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:06.275209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:06.276043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:06.276043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:06.276043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:06.280073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 254/497] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:06.280912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.297984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5776d-85d4-486a-b360-d601e7ceb118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.298611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5776d-85d4-486a-b360-d601e7ceb118 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.298889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5776d-85d4-486a-b360-d601e7ceb118 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.299155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5776d-85d4-486a-b360-d601e7ceb118 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.306400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5776d-85d4-486a-b360-d601e7ceb118 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.307824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5776d-85d4-486a-b360-d601e7ceb118 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.310583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.314481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-47e5776d-85d4-486a-b360-d601e7ceb118 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:06.321869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:06.322776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:06.323647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 244/498] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:06.324230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabcab0-3084-43ef-8e8d-913d9f354b90 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.324938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabcab0-3084-43ef-8e8d-913d9f354b90 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.324938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabcab0-3084-43ef-8e8d-913d9f354b90 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.330292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 245/499] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:06.331052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabcab0-3084-43ef-8e8d-913d9f354b90 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.396514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a31043-a221-423f-bee0-1b7c9d865534 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.430596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.431216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.431499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.431774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.435213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabcab0-3084-43ef-8e8d-913d9f354b90 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.442083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.443548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:06.457898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:06.458950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:06.459790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:06.459790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:06.459790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:06.459790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 255/500] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:06.459790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.471521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42b9b5-9f2c-46ff-bda1-191c63aa6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.472122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b9b5-9f2c-46ff-bda1-191c63aa6468 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.472437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b9b5-9f2c-46ff-bda1-191c63aa6468 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.472707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b9b5-9f2c-46ff-bda1-191c63aa6468 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.473344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42b9b5-9f2c-46ff-bda1-191c63aa6468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:06.473648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42b9b5-9f2c-46ff-bda1-191c63aa6468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.475011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42b9b5-9f2c-46ff-bda1-191c63aa6468 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.486692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 246/501] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:06.487881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611163d9-dc04-421b-81c1-6b0c2d2b774e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.488158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611163d9-dc04-421b-81c1-6b0c2d2b774e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.488461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611163d9-dc04-421b-81c1-6b0c2d2b774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.506471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-611163d9-dc04-421b-81c1-6b0c2d2b774e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:06.508150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-611163d9-dc04-421b-81c1-6b0c2d2b774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:06.538968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.558788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84a014-b5cc-4fc3-b351-5a04f2604f03 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.563429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.575472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 247/502] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:06.576292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611163d9-dc04-421b-81c1-6b0c2d2b774e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.591923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.592566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.593115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.593457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.594197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:06.594707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.595975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.639965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 256/503] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:06.639965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.682736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611163d9-dc04-421b-81c1-6b0c2d2b774e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.718304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.719006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.719283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.719558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.729932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:06.734787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:06.735630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:06.741792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:06.742533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5318f5-27e1-49af-854d-2f71c855f5bd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.742934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 248/504] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:06.742934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.775683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.776326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.776668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.777005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.777747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:06.778109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.779570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.786292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.792135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 257/505] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:06.792964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.803551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.856635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54998a4-d3cb-41b3-8eca-8834b27ac257 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.889440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.890057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.890331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.890597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.891513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95570723-983a-4039-9df8-031f5ce98a04 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:06.897424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.898868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:06.910111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:06.911017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 249/506] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:06.911883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:06.928093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:06.929384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:06.929663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:06.929941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:06.991697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:06.995925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:06.997420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.008745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ca10-c03f-4ffe-9c1d-2287003b3ae6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.018720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.042121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.042649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.042925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.043194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.043843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:07.044141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.045514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.053090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 250/507] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:07.054408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.093170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 258/508] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:07.093990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.152034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5438a97-a500-4085-81c6-f8898701b153 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.185764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.186379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.186652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.186920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.191343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.194877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.195191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8210f5-653d-4e64-9407-e4dd50089f6f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.196347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:07.207223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:07.208280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 251/509] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:07.209257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.233422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.234039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.234314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.234587 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.239609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.244923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.246366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:07.260591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:07.261608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:07.262639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:07.262639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:07.262639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:07.264381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 259/510] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:07.265179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.310046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1e65e-f760-40d3-8239-8b8bac5bba51 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.342918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.343517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.343793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.344063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.344813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:07.345116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.346541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.365799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2683c-335c-4ec2-ae66-4e0e4a91fba8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.382270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 252/511] 10.222.0.50 () {56 vars in 1137 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:07.383085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.400833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.401462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.401739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.402008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.412165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.414169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:07.427776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:07.428879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:07.429935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:07.429935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:07.429935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:07.429935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 260/512] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:07.430197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.432503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.473921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.484017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e137c1c-da5b-4381-a69c-ffdba3106d01 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.516857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.517553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.517830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.518098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.528409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db16f7-1312-402c-a8ab-ac64d9cbc97e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.528674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.530308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:07.547316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:07.548364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:07.553713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:07.553713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:07.553713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:07.553713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 253/513] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:07.553713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.562293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.562885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.563159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.563428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.564146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:07.564476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.565905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.582816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 261/514] 10.222.0.50 () {56 vars in 1137 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:07.583606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.641099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.646307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bec7c1-a750-48b0-a771-da6cc4ef6132 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.679491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.680022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f6c05-045b-465c-8e50-ed6b9c57ac54 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.680120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.680389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.680668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.681292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:07.681601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.682968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.695229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 254/515] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:07.696023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.698705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.714916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.715516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.715793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.716068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.782518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.783977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.795166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894ed6-8e59-42e6-bbac-6cba21730272 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.829080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.829622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.829899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.830173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.830808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:07.831113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.832503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.839203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.844003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 255/516] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:07.844903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.879332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 262/517] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:07.880215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:07.905087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:07.944108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f38db3-be11-4989-b4da-e41290d0ea13 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.977319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792d44e-b6a2-494c-bee0-75db1a3fdf26 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:07.977623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:07.978226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:07.978561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:07.978840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:07.979485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:07.979787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:07.981219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:07.993760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 256/518] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:07.993760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.015018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.015610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.015884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.016151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.042828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.083599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.085125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.091240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcb4d4-b2da-43fc-b39b-908eae1da44d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.112769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.124517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.125109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.125380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.125645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.126278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.127658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.149312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 257/519] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:43:07 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:08.150422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.189654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 263/520] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:08.190845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.264275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.287749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927623ed-cf07-4a53-bd05-8921a6985ee0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.317432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c56eaa-4761-44a6-a930-b433c901173b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.331330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.331706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.331986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.332296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.340937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.342348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:08.354121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:08.355178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:08.356310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 258/521] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:08.360631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.361587 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.362318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.362685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.363050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.374305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.375763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:08.391067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:08.392010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:08.393230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:08.393230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:08.393230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:08.393230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 264/522] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:08.393230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.410756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.475986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb87759-9c2c-41d2-83dc-1ebbfa755626 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.510234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7573d-4561-40fb-92d2-2499fc15d432 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.511193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.511820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.512094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.512395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.518945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.520407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.525885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:08.533536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:08.534396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:08.535347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 259/523] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:08.536536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.550794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.551535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.551807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.552150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.563967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.565570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:08.581626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:08.582542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:08.583387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:08.583387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:08.583387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:08.583387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 265/524] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:08.583617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.642630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.648272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23b98d-bc95-439a-b407-5b2704228be5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.684353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.684968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.685238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.685530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.686203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:08.686505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.687938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.691806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a7cf4-f022-4d49-9c16-f75edb985529 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.706610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 260/525] 10.222.0.50 () {56 vars in 1139 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:08.707429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.732350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.732955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.733245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.733525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.734170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:08.734476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.735843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.752805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 266/526] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:08.752805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.775129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.817299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60955b-4413-4ccd-9200-79744c08c5f1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.853465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.854072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.854348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.854615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.855297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:08.855599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.857041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.857846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa86357-0024-4290-8649-58dca7eaf386 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:08.879682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 261/527] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:08.879682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.895012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:08.901816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:08.902418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:08.902692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:08.902966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:08.903592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:08.903886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:08.905273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:08.921123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 267/528] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:08.921123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:08.982690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58593aa-f7ca-4f64-8748-25adfe22e3f2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.002203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.018625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.019237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.019520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.019796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.022375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a00561-adc5-467f-b9e9-9fa97880626c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.058312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.058933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.059207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.059478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.067441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.068932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:09.080925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:09.081750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 268/529] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:09.087601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.098695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.100140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.119287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.196931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e7996-1588-4292-97fb-bf07630db332 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.204041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 262/530] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:09.205037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.234941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.235550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.235831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.236400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.237110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=95b365b4794c4edebe306e09336c42f3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:09.237422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.238867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.256288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 269/531] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/role_assignments?user.id=95b365b4794c4edebe306e09336c42f3&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:09.256288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.269343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.334234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071e54e-3b25-40d7-a22d-a9b7639dbc07 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.353805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.376869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.377657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.378107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.378472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.386436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.388209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:09.401153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:09.402003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:09.402823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 263/532] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:09.403730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.405707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53263c28-a3d8-4481-adfa-8959275b4fe0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.447275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.447779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.448051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.448346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.449035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:09.449339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.450780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.474206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 270/533] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:09.475168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.530642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6ff47-8556-4e9a-80c6-904f391fc6a1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.558155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.567896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.568833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.569280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.569650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.581116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.582768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:09.597734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:09.598816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:09.599830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:09.599830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:09.599830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:09.606114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 264/534] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:09.606114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.606346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6169e-e398-4547-9a4c-fc041cb2ab7b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.640460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.650898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.651816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.652334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.652805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.653699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:09.654197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.656374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.673632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 271/535] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:09.675144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.719974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e740d6d-8209-4449-9525-7a504dc01054 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.757201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.758043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.758331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.758602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.759325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=b68fc66dbef44519a2212d35655bf6a7) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:09.759627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.761229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.769013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 265/536] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/role_assignments?user.id=b68fc66dbef44519a2212d35655bf6a7&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:09.769833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.809922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4746542-83f6-42c6-a2c8-0e586b6dc4d7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:09.844342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.860644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:09.863401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:09.863901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:09.864421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:09.865277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:09.865788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:09.867426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:09.883425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 272/537] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:09.905697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:09.982138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:09.993480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad529-62b8-4611-ab34-7993d33f44f7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:10.038938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:10.039927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:10.040424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:10.040965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:10.041915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=88273fe4807b4f08911298b7bf0c3653) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:10.042412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:10.044041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:10.056330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 266/538] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/role_assignments?user.id=88273fe4807b4f08911298b7bf0c3653&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:10.058889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a428e-aac0-4e82-8a43-891c26dc0717 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:10.098465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ae18f-590f-438b-952d-53fc829251bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:10.099100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ae18f-590f-438b-952d-53fc829251bb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:10.099374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ae18f-590f-438b-952d-53fc829251bb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:10.099641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ae18f-590f-438b-952d-53fc829251bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:10.106969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ae18f-590f-438b-952d-53fc829251bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:10.108462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ae18f-590f-438b-952d-53fc829251bb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-696ae18f-590f-438b-952d-53fc829251bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:10.120888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:10.121785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:10.122602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 273/539] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:10.127922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:10.138979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339900c9-0620-4eb2-af5c-b14e783f6036 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:10.139301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339900c9-0620-4eb2-af5c-b14e783f6036 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:10.139548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339900c9-0620-4eb2-af5c-b14e783f6036 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:10.142351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 274/540] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:10.146726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82f5f-303c-42ce-9d47-e1218cbff01a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:10.147009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82f5f-303c-42ce-9d47-e1218cbff01a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:10.147898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82f5f-303c-42ce-9d47-e1218cbff01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:10.180115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-11f82f5f-303c-42ce-9d47-e1218cbff01a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:10.181361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-11f82f5f-303c-42ce-9d47-e1218cbff01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:10.264740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:10.305469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c435fd-2b3a-4c19-86ad-c0695d507d1b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:10.316624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 275/541] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:43:10.357121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:10.357736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:10.358015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:10.358288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:10.358988 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:10.359293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:10.360782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:10.374503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 267/542] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:10.381396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82f5f-303c-42ce-9d47-e1218cbff01a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:10.516338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82f5f-303c-42ce-9d47-e1218cbff01a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:10.551736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:10.552375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:10.552656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:10.552927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:10.554218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=e1c3b7f7c83f48aca27fbd46202f6506) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:10.554526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:10.555985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:10.562022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 276/543] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:10 2022] GET /identity/v3/role_assignments?user.id=e1c3b7f7c83f48aca27fbd46202f6506&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:10.630653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:10.658381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:10.764966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0359c0e-458a-4b25-846f-b99f6b2b32cd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:10.812125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:10.813119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:10.813628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:10.814091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:10.833354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:10.838700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:10.839548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:10.840348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:10.842619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 268/544] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:10.848736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:10.899070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:10.963520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14f090-8a22-40a7-bef4-baceefdc8945 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:11.012418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:11.013032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:11.013309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:11.013581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:11.014242 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:11.014542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:11.015903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:11.024792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 277/545] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:11.030242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:11.136371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:11.169473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116bffb-3a2a-4b4d-a481-51a4fefd237a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:11.221837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:11.222480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:11.222758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:11.223029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:11.241673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:11.243162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:11.261596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:11.262545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:11.263346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:11.263346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:11.263346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:11.263346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 269/546] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:11.269399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:11.388655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe0b8-0e0c-497a-9d3d-a289d98cb43e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:11.390904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:11.434306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd0d17-9424-4e83-aaa3-c5be4d034cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:11.434917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd0d17-9424-4e83-aaa3-c5be4d034cce None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:11.435193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd0d17-9424-4e83-aaa3-c5be4d034cce None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:11.435466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd0d17-9424-4e83-aaa3-c5be4d034cce None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:11.436109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd0d17-9424-4e83-aaa3-c5be4d034cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:11.436443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd0d17-9424-4e83-aaa3-c5be4d034cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:11.437809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd0d17-9424-4e83-aaa3-c5be4d034cce None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:11.454269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 278/547] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:11.459341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:11.486100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c61f-91da-49bc-a9bc-38c5f038370c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:11.486641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c61f-91da-49bc-a9bc-38c5f038370c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:11.486923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c61f-91da-49bc-a9bc-38c5f038370c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:11.488550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 279/548] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:11.493311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d626af-b422-41fe-9819-62fa86fd1e6a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:11.493592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d626af-b422-41fe-9819-62fa86fd1e6a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:11.493861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d626af-b422-41fe-9819-62fa86fd1e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:11.513750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-47d626af-b422-41fe-9819-62fa86fd1e6a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:11.514933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-47d626af-b422-41fe-9819-62fa86fd1e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:11.603197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 280/549] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:43:11.631355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ac4e4-c4e6-4eca-8c0c-525226b4295a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:11.665899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:11.679060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:11.679698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:11.679975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:11.680280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:11.688160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:11.689622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:11.707294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:11.708744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:11.709826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:11.712147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 270/550] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:11.717342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d626af-b422-41fe-9819-62fa86fd1e6a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:11.852613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:11.860991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d626af-b422-41fe-9819-62fa86fd1e6a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:11.907294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:11.907910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:11.908168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:11.908466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:11.909194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:11.909496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:11.910943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:11.930059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 281/551] 10.222.0.50 () {56 vars in 1135 bytes} [Tue Aug 30 01:43:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:11.935287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:11.939254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.045857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0cfa-dc4e-49c1-ad87-8928e2bf2186 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:12.080496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02c918-cc98-4b4f-a886-84ee9e19ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:12.081134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02c918-cc98-4b4f-a886-84ee9e19ef61 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.081414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02c918-cc98-4b4f-a886-84ee9e19ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.081693 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02c918-cc98-4b4f-a886-84ee9e19ef61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.126899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7263f45-2c11-4d7d-96ce-c86a9f094d33 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:12.141526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:12.152942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02c918-cc98-4b4f-a886-84ee9e19ef61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:12.154499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02c918-cc98-4b4f-a886-84ee9e19ef61 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:12.207674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d2db0-5605-429d-9201-06113ccb2d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:12.208578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d2db0-5605-429d-9201-06113ccb2d96 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.208998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d2db0-5605-429d-9201-06113ccb2d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.209419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d2db0-5605-429d-9201-06113ccb2d96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.241419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d2db0-5605-429d-9201-06113ccb2d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c29d2db0-5605-429d-9201-06113ccb2d96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:12.248889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:12.249888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:12.250707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:12.253844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 282/552] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:12.255909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290090b-74ae-492b-aa88-e9cd3e07f287 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.256549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290090b-74ae-492b-aa88-e9cd3e07f287 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.257570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290090b-74ae-492b-aa88-e9cd3e07f287 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.263803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 283/553] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:12.263803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290090b-74ae-492b-aa88-e9cd3e07f287 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.283916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:12.319612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 271/554] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:12.321827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca003e5-ef52-489d-a53a-20fda3e69bfc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.322314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca003e5-ef52-489d-a53a-20fda3e69bfc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.322783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca003e5-ef52-489d-a53a-20fda3e69bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.347647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9ca003e5-ef52-489d-a53a-20fda3e69bfc None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:12.349479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9ca003e5-ef52-489d-a53a-20fda3e69bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:12.394195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290090b-74ae-492b-aa88-e9cd3e07f287 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:12.430927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 272/555] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:12.432037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca003e5-ef52-489d-a53a-20fda3e69bfc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.438691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c30fc0-3c0b-40f2-9544-6036d608cdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:12.440905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c30fc0-3c0b-40f2-9544-6036d608cdab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.441394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c30fc0-3c0b-40f2-9544-6036d608cdab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.442369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c30fc0-3c0b-40f2-9544-6036d608cdab None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.443831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c30fc0-3c0b-40f2-9544-6036d608cdab None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:12.445160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c30fc0-3c0b-40f2-9544-6036d608cdab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:12.446604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c30fc0-3c0b-40f2-9544-6036d608cdab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:12.457436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:12.462514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 284/556] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:12.463772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0277d-09ec-4885-9b61-86725cac67c3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.464066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0277d-09ec-4885-9b61-86725cac67c3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.464395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0277d-09ec-4885-9b61-86725cac67c3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.465803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 285/557] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:12.466346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0277d-09ec-4885-9b61-86725cac67c3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.558508 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:12.561337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca003e5-ef52-489d-a53a-20fda3e69bfc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:12.592778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0277d-09ec-4885-9b61-86725cac67c3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:12.600537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc905c-da1a-4316-9c6d-7cdbd82ef5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:12.601175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc905c-da1a-4316-9c6d-7cdbd82ef5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.601453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc905c-da1a-4316-9c6d-7cdbd82ef5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.601722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc905c-da1a-4316-9c6d-7cdbd82ef5e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.612976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc905c-da1a-4316-9c6d-7cdbd82ef5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:12.614524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc905c-da1a-4316-9c6d-7cdbd82ef5e6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:12.629509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:12.630188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.630464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.630789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-85dc905c-da1a-4316-9c6d-7cdbd82ef5e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:12.630906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:12.632312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:12.634220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:12.634220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:12.634220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:12.634220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 273/558] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:12.634220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17bf18-33c5-4bae-8bf6-4b6082dcc68e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.634220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17bf18-33c5-4bae-8bf6-4b6082dcc68e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.635037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17bf18-33c5-4bae-8bf6-4b6082dcc68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.645127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:12.646718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:12.652969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9c17bf18-33c5-4bae-8bf6-4b6082dcc68e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:12.654252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9c17bf18-33c5-4bae-8bf6-4b6082dcc68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:12.662979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:12.665033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:12.667037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:12.667037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:12.667037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:12.667037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 286/559] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:12.667037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.733009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:12.742288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 274/560] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:12.743130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17bf18-33c5-4bae-8bf6-4b6082dcc68e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.789460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032f8a8-b34f-42e8-926a-190b2d4e454e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:12.818837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:12.829786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:12.830811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.831285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.831936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.832869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:12.833364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:12.835592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:12.848851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 287/561] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:12.850073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.872605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17bf18-33c5-4bae-8bf6-4b6082dcc68e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:12.914683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:12.915292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:12.915570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:12.915847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:12.916520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:12.916823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:12.918765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:12.936528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 275/562] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:12.938091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:12.971351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906e140-39ac-4449-98f0-e2f0b10bd7be None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.008573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.009629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.010100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.010572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.017509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.023087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:13.029957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:13.030876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 288/563] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:13.036596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.061448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c02f1-2f7b-4da5-a50a-10ceeab7ea13 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.087391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.096236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.096884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.097167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.097441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.103893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.105350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:13.116449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:13.117349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:13.118218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 276/564] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:13.118928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.143159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784e2ac-ff37-4135-8164-966a61a81c0f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.176045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c1839-8d14-494a-9a08-28978767267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.176709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c1839-8d14-494a-9a08-28978767267a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.176993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c1839-8d14-494a-9a08-28978767267a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.177269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c1839-8d14-494a-9a08-28978767267a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.185043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c1839-8d14-494a-9a08-28978767267a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.186484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c1839-8d14-494a-9a08-28978767267a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-221c1839-8d14-494a-9a08-28978767267a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:13.199665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:13.200641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 289/565] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:13.201614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c1839-8d14-494a-9a08-28978767267a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.219495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4024601-6251-4d30-87c3-6bb2c03dbc45 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.238693 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.253073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec6c38-94c6-42a4-9799-3f8355cc4833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.253692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec6c38-94c6-42a4-9799-3f8355cc4833 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.253965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec6c38-94c6-42a4-9799-3f8355cc4833 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.254234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec6c38-94c6-42a4-9799-3f8355cc4833 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.264713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec6c38-94c6-42a4-9799-3f8355cc4833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-58ec6c38-94c6-42a4-9799-3f8355cc4833 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:13.269905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:13.270996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:13.271998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 277/566] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:13.272916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed30915-e670-46d8-9960-72d9a07d9d10 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.272916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed30915-e670-46d8-9960-72d9a07d9d10 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.272916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed30915-e670-46d8-9960-72d9a07d9d10 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.277706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 278/567] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:13.277706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed30915-e670-46d8-9960-72d9a07d9d10 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.301294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c1839-8d14-494a-9a08-28978767267a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.308831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.335669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.336324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.336621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.336891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.337532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:13.337833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.339197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.346859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 290/568] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:13.347460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.384225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed30915-e670-46d8-9960-72d9a07d9d10 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.415757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.416382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.416660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.416931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.417612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:13.417911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.419323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.431193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 279/569] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:13.431960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.444957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2a01a-0a9b-4e32-adfa-75d65902c73c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.473594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.477322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd820ff-88f1-4ea2-b76d-8719727c953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.477950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd820ff-88f1-4ea2-b76d-8719727c953a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.478224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd820ff-88f1-4ea2-b76d-8719727c953a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.478493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd820ff-88f1-4ea2-b76d-8719727c953a None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.479204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd820ff-88f1-4ea2-b76d-8719727c953a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:13.479502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd820ff-88f1-4ea2-b76d-8719727c953a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.480944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd820ff-88f1-4ea2-b76d-8719727c953a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.498362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 291/570] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:13.499512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1863d-d5cf-446a-b738-f531249ac8b8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.499792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1863d-d5cf-446a-b738-f531249ac8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.500067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1863d-d5cf-446a-b738-f531249ac8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.517336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-69b1863d-d5cf-446a-b738-f531249ac8b8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:13.518443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-69b1863d-d5cf-446a-b738-f531249ac8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:13.531112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19347ad5-2398-49c5-a816-524a1a75b160 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.537622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.564716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.565311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.565592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.565858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.566503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:13.566802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.568139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.575693 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 280/571] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:13.576494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.588199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 292/572] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:13.589102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1863d-d5cf-446a-b738-f531249ac8b8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.684423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6cf83-35d5-4053-80cf-3a86b1b558ad None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.686153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.691039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1863d-d5cf-446a-b738-f531249ac8b8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.721958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.722687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.723036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.723446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.725854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.726524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.726874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.727147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.727867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=5956fdd8d6b54640970cde2705a83260) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:13.728172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.729630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.740397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 293/573] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/role_assignments?user.id=5956fdd8d6b54640970cde2705a83260&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:13.740681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.740771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.740844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.765646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:13.768779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:13.769706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:13.770565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:13.770565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:13.770565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:13.772412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 281/574] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:13.773310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.885817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aa5a3-fa81-4f32-a671-d7f5afe25487 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.896531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff237f2-07f0-4c32-858d-b1e02c79cf02 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:13.922970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.923604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.923881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.924152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.933680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:13.934302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:13.934580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:13.934852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:13.935675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.937147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.946620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:13.948065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:13.951184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:13.952106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:13.953294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:13.953294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:13.953294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:13.953294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 294/575] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:13.957648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:13.963590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:13.964727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:13.965582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:13.965582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:13.965582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:13.965582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 282/576] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:13.966085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:13.975990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.040666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.071879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd266-d2ee-4a30-bdaa-76bd1e07f5d3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.076170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68d390-ea86-4de2-9530-b4f978aabcc0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.107754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.108671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.109102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.109374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.110054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:14.110304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.111913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.112702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.113578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.113851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.114645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.123881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 295/577] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:14.124705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.131189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:14.136620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:14.137503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:14.138321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 283/578] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:14.139916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.219705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.231971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228f9ca-e0a8-43a1-949e-e06defbfc508 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.258079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07bd54-4812-4bf5-9d2e-30be65e60617 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.261499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.266844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.267447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.267983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.267983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.268643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:14.268944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.270315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.286299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 296/579] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:14.286299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.306210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.306852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.307131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.307413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.308040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:14.308383 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.309731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.322197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 284/580] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:14.323069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.398615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb7d53-6acd-4b3a-8211-93df72092836 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.429868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a94b-8f50-4544-9e07-077da3c16a3c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.431989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.432636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.432911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.433180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.440388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:14.453151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:14.454036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 297/581] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:14.454956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.462025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.471741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.472373 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.472860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.472997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.473544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:14.473936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.475170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.484320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 285/582] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:14.484320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.519570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.574613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad39dd2-2874-4563-8ee4-fa5c6183ffe1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.590048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ed298-f61f-4de2-8ef1-804e85bea867 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.609909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.610531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.610808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.611061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.618482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.620067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.623440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.624067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.624380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.624655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:14.631372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:14.632804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:14.638469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:14.638895 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.638967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 298/583] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:14.638967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.639096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:14.650652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:14.651567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 286/584] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:14.653405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.706953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.748582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d3cd5-b860-4c80-9681-ca4726eebce1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.761657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.765665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333b63e-acdd-412c-b3b2-2defb6b57f07 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.785153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.785761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.786152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.786424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.787158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:14.787458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.788952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.801188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.801837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.802121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.802406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.807646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 299/585] 10.222.0.50 () {56 vars in 1137 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:14.808911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.816204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.817749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:14.833333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:14.834266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:14.835187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:14.835187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:14.835187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:14.835187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 287/586] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:14.835187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.921579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b200-ade9-4a6b-95bf-1b98e55c787a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.936723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:14.939685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859b357-9dd1-45db-b391-00aa4aaa18a9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:14.961731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.962591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.962869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.963297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.964382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:14.964934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.966532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.974050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:14.974659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:14.974939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:14.975214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:14.975956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:14.976283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:14.977683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:14.981776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 300/587] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:14.982745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:14.995771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 288/588] 10.222.0.50 () {56 vars in 1137 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:14.996686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.022457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:15.096965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e4a5-a8af-4cbf-8e59-6a15aab5f7a3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.101000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e724-47be-4abf-b371-8c267ff1340b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.133157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.133873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.134228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.134506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.136134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.136788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.137063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.137336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.137975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:15.138273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.139616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.147240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.149728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.150789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 289/589] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:15.151567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:15.166288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:15.167360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:15.168213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:15.168213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:15.168213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:15.173538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 301/590] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:15.173538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.175392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:15.259597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f8c5c-c710-4349-86c1-22e901688cda None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.259829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:15.278778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb22f1-680c-486a-a069-0387dc899141 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.301961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.302775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.303050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.303322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/95b365b4794c4edebe306e09336c42f3/roles/3aa65bdcfd1244cb9d358fa12c4c02cc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.315412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.316091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.316406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.316763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.318933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=95b365b4794c4edebe306e09336c42f3, role_id=3aa65bdcfd1244cb9d358fa12c4c02cc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.320465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.328032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.329557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.333221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 290/591] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:15 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/95b365b4794c4edebe306e09336c42f3/roles/3aa65bdcfd1244cb9d358fa12c4c02cc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:15.337136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:15.347923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:15.348971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:15.349854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:15.349854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:15.349854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:15.349854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 302/592] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:15.351069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.438285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:15.476547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a031fe-02ec-4abc-b2ae-0c8f4aae6d15 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.490208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e519e-8341-43ef-aa73-0d6b9b4a53ef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.523156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.523791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.524067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.524380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.530723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.531773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.532398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.532680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.533072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.533915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:15.534114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.534762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.536515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:15.548429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:15.549389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:15.550303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 291/593] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:15.550876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.553444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 303/594] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:15.553591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:15.556561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.675766 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ee8c4-1564-4eec-9be2-934685010a0e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.675953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cd18e-31cf-4e2a-99b3-c01f2ff6eab6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.702863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:15.711112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1980c-fb98-4062-92b8-66c82a824056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.711720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1980c-fb98-4062-92b8-66c82a824056 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.711996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1980c-fb98-4062-92b8-66c82a824056 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.712292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1980c-fb98-4062-92b8-66c82a824056 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.712556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.713020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1980c-fb98-4062-92b8-66c82a824056 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:15.713179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.713329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1980c-fb98-4062-92b8-66c82a824056 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.713419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.713680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.714308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:15.714606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.714758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1980c-fb98-4062-92b8-66c82a824056 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.715931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.729839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 292/595] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:15.737656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 304/596] 10.222.0.50 () {56 vars in 1139 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:15.737656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b28bf-5049-4aca-b861-23254c46710a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.737656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b28bf-5049-4aca-b861-23254c46710a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.737908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.737979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b28bf-5049-4aca-b861-23254c46710a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.737979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 305/597] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:15.737979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b28bf-5049-4aca-b861-23254c46710a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.796335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:15.846953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b28bf-5049-4aca-b861-23254c46710a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.851334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7645414-3a24-409b-8c8e-e20644d9bc8c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:15.884722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.885354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.885630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.885898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.886369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a16e1-4246-48f7-94a1-5ac202093e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:15.887149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a16e1-4246-48f7-94a1-5ac202093e44 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.887425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a16e1-4246-48f7-94a1-5ac202093e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.887702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a16e1-4246-48f7-94a1-5ac202093e44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.892776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.894210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.894328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a16e1-4246-48f7-94a1-5ac202093e44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:15.895846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a16e1-4246-48f7-94a1-5ac202093e44 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:15.906900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:15.907866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:15.913695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8a9a16e1-4246-48f7-94a1-5ac202093e44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:15.914119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:15.914904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:15.917472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:15.917910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 306/598] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:15.917910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:15.918043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 293/599] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:15.918043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f3992-f743-4a68-b9fe-054461a2b09d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:15.918043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f3992-f743-4a68-b9fe-054461a2b09d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:15.918043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f3992-f743-4a68-b9fe-054461a2b09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:15.933878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e39f3992-f743-4a68-b9fe-054461a2b09d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:15.934980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e39f3992-f743-4a68-b9fe-054461a2b09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:15.981160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.007169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 294/600] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:16.008074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f3992-f743-4a68-b9fe-054461a2b09d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.020312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4de3a-0eb1-4cd7-8611-0e0e70888574 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.036154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.060512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.061428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.061887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.062442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.074121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.075753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:16.092294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:16.093405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:16.094495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:16.094495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:16.094495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:16.095378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 307/601] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:16.096181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.139432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f3992-f743-4a68-b9fe-054461a2b09d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.182868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.183688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.184087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.184412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.185289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:16.185706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.187213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.204695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 295/602] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:16.205804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.210014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa003507-4511-4a31-a939-74b5be3e5926 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.249939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.250842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.252479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.252988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.253620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.256577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:16.256577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.256577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.271974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 308/603] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:16.273601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.300414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.320577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1dc5a-d0ff-4961-83e6-95659643ca9e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.356904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.357512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.357788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.358059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.358697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:16.358996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.360379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.373017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 296/604] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:16.373811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.405411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01540939-4062-4cd4-ae45-1dc17f3cd242 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.440840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.441514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.441866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.442209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/88273fe4807b4f08911298b7bf0c3653/roles/3aa65bdcfd1244cb9d358fa12c4c02cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.458678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=88273fe4807b4f08911298b7bf0c3653, role_id=3aa65bdcfd1244cb9d358fa12c4c02cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.460213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.473668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 309/605] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:16 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/88273fe4807b4f08911298b7bf0c3653/roles/3aa65bdcfd1244cb9d358fa12c4c02cc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:16.474499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.490125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.497835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5963b5-5ce9-4e59-9bca-1f834f26dfd7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.560578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.562163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.562641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.563326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/b68fc66dbef44519a2212d35655bf6a7/roles/3aa65bdcfd1244cb9d358fa12c4c02cc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.590233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=b68fc66dbef44519a2212d35655bf6a7, role_id=3aa65bdcfd1244cb9d358fa12c4c02cc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.591581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.593482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.614330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 297/606] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:16 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/b68fc66dbef44519a2212d35655bf6a7/roles/3aa65bdcfd1244cb9d358fa12c4c02cc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:16.616788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.626951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a94255-dd2e-43fe-84c3-edb319b7f375 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.680620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.681435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.681814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.682194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.694730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:16.700150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:16.701310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:16.702366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:16.704816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 310/607] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:16.717372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.753659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec681b5-c7cc-4c9c-8391-911cbb548f3d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.794662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.800273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.800273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.800273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.810956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.812458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:16.826582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:16.827521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:16.828704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 298/608] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:16.833740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:16.845277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.893381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed0f72-fee0-429f-804f-7650554ec213 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.941797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:16.942777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:16.943473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:16.944373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:16.945534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:16.946014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:16.947036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2856e6-9359-408f-8113-d845c255a045 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:16.947783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:16.950564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:16.961750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 311/609] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:16.966896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:17.036993 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:17.037620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.038465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.038740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.039441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:17.039749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:17.041202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:17.063484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 299/610] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:17.069123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:17.085123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdbc1c-9907-4832-a623-f6589b10ed54 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:17.124579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:17.125056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.125335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.125611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.136971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:17.138492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:17.152833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:17.153763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:17.155026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:17.155026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:17.155026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:17.155026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 312/611] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:17.155557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:17.159664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:17.201968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a9a5-c1f0-46e9-bbc0-e4b1264954ba None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:17.213246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:17.248534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:17.249551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.250026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.250512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/e1c3b7f7c83f48aca27fbd46202f6506/roles/3aa65bdcfd1244cb9d358fa12c4c02cc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.267175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=e1c3b7f7c83f48aca27fbd46202f6506, role_id=3aa65bdcfd1244cb9d358fa12c4c02cc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:17.269065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:17.293025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a28c2-9af4-414a-a52c-16369de43ac4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:17.306142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 300/612] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:17 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/e1c3b7f7c83f48aca27fbd46202f6506/roles/3aa65bdcfd1244cb9d358fa12c4c02cc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:43:17.339142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2111879-f23a-435b-a362-c2dab9f7a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:17.340136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111879-f23a-435b-a362-c2dab9f7a771 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.340664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111879-f23a-435b-a362-c2dab9f7a771 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.341193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111879-f23a-435b-a362-c2dab9f7a771 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.342099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111879-f23a-435b-a362-c2dab9f7a771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:17.342644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111879-f23a-435b-a362-c2dab9f7a771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:17.344455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111879-f23a-435b-a362-c2dab9f7a771 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:17.358954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 313/613] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:17.364800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:17.496109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536ccd-4f40-43a0-ba5e-8b0fc20b6c04 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:17.518063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc295-fb01-41e9-99a1-3c9e8d0ac3cf None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.518384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc295-fb01-41e9-99a1-3c9e8d0ac3cf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.518628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fc295-fb01-41e9-99a1-3c9e8d0ac3cf None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.520279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 314/614] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:17.525950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c94aaa-85c2-4759-b6c9-a5492a9c55aa None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.526236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c94aaa-85c2-4759-b6c9-a5492a9c55aa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.526496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c94aaa-85c2-4759-b6c9-a5492a9c55aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.530644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:17.555179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-24c94aaa-85c2-4759-b6c9-a5492a9c55aa None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:17.558286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-24c94aaa-85c2-4759-b6c9-a5492a9c55aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:17.561391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:17.562354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.562827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.563291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.579518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:17.583028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:17.612300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:17.613269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:17.614164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:17.617550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 301/615] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:17.624742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:17.682335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 315/616] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:43:17.785751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7ed8c-7cfe-48d4-9e78-e7a9b03eb03e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:17.828382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:17.829389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:17.829866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:17.830390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:17.831316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:17.831813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:17.833515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:17.859524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 302/617] 10.222.0.50 () {56 vars in 1135 bytes} [Tue Aug 30 01:43:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:17.865255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c94aaa-85c2-4759-b6c9-a5492a9c55aa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:17.945350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:18.013028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c94aaa-85c2-4759-b6c9-a5492a9c55aa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.026518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.076483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.077401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:18.077882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:18.078328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:18.093564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:18.094793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:18.100094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:18.124411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:18.125411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:18.126277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:18.126277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:18.126277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:18.130735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 316/618] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:18.140888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.221133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8de9b4-4b93-4f00-856f-37bd48729f55 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.259587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.260443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:18.260817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:18.261185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:18.276403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:18.277166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:18.278040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:18.279013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 303/619] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:18.299324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.318644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:18.327408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee11f-81a2-4f1e-8d49-b5d16c526b9f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.380281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.380281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:18.380281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:18.380281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:18.380940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:18.380940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:18.381989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:18.395674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 317/620] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:18.401921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.476345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:18.482846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0bdf-d1e2-472c-8140-898275ec210f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.521968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.522611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:18.522884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:18.523152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:18.528265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:18.528265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:18.528265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:18.539474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 304/621] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:18.545013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.609976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e105996-f408-4e4d-b4b6-994f0189ef7c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.649662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.650298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:18.650575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:18.650847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:18.661013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:18.662527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:18.687058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:18.688011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:18.688919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:18.695270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 318/622] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:18.698854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.720592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:18.742633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef14f4-000f-4fcf-9a7f-03d388b1e63e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.776758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.777311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:18.777588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:18.777858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:18.794643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:18.795859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:18.822806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:18.823778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:18.824682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:18.824682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:18.824682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:18.832786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 305/623] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:18.838000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.840393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4b40e-867e-476f-b65a-1c3c444104be None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.864669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:18.896952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.897982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:18.898466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:18.898939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:18.899936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:18.900501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:18.902134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:18.924984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 319/624] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:18.931288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:18.963880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec7a2-599b-47be-acc7-d4ff4930d3d9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:18.999147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb821a44-4bd9-4388-90cc-14873bf6490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:18.999814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821a44-4bd9-4388-90cc-14873bf6490d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.000092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821a44-4bd9-4388-90cc-14873bf6490d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.000389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821a44-4bd9-4388-90cc-14873bf6490d None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.001024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb821a44-4bd9-4388-90cc-14873bf6490d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:19.001321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb821a44-4bd9-4388-90cc-14873bf6490d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:19.002664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb821a44-4bd9-4388-90cc-14873bf6490d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:19.014162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:19.020149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 306/625] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:19.020149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe84568-5d02-4d21-87b1-9e6871a97e65 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.020149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe84568-5d02-4d21-87b1-9e6871a97e65 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.020149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe84568-5d02-4d21-87b1-9e6871a97e65 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.020149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 307/626] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:19.020149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe84568-5d02-4d21-87b1-9e6871a97e65 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:19.060083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcee0a1-831d-42e0-93fd-895e5ca77bbb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:19.093009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:19.098537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b79df2-41a5-428d-a6f1-7e7869a48d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:19.099400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79df2-41a5-428d-a6f1-7e7869a48d71 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.099851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79df2-41a5-428d-a6f1-7e7869a48d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.100859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79df2-41a5-428d-a6f1-7e7869a48d71 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/5956fdd8d6b54640970cde2705a83260/roles/3aa65bdcfd1244cb9d358fa12c4c02cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.120651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b79df2-41a5-428d-a6f1-7e7869a48d71 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=5956fdd8d6b54640970cde2705a83260, role_id=3aa65bdcfd1244cb9d358fa12c4c02cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:19.123046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b79df2-41a5-428d-a6f1-7e7869a48d71 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:19.125514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe84568-5d02-4d21-87b1-9e6871a97e65 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:19.144007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 320/627] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:18 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/5956fdd8d6b54640970cde2705a83260/roles/3aa65bdcfd1244cb9d358fa12c4c02cc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:19.145474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbeb01-211e-4ef4-ae5f-f6d25fba2feb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.145756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbeb01-211e-4ef4-ae5f-f6d25fba2feb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.146078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbeb01-211e-4ef4-ae5f-f6d25fba2feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.170717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ccdbeb01-211e-4ef4-ae5f-f6d25fba2feb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:19.171932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ccdbeb01-211e-4ef4-ae5f-f6d25fba2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:19.176208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6dec-4c72-4aba-a940-7e5e32384874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:19.177297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6dec-4c72-4aba-a940-7e5e32384874 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.177796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6dec-4c72-4aba-a940-7e5e32384874 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.178273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6dec-4c72-4aba-a940-7e5e32384874 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.189443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6dec-4c72-4aba-a940-7e5e32384874 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:19.192339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6dec-4c72-4aba-a940-7e5e32384874 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-274a6dec-4c72-4aba-a940-7e5e32384874 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:19.211063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:19.212371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:19.213343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:19.214082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 308/628] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:43:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:19.215589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65299-53d2-40f7-9825-ec0e0c105c1b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.215937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65299-53d2-40f7-9825-ec0e0c105c1b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.216316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df65299-53d2-40f7-9825-ec0e0c105c1b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.220387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 309/629] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:19.221583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65299-53d2-40f7-9825-ec0e0c105c1b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:19.262911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 321/630] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:19.264593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50902d6-6b9c-4a5e-b166-b3f34a4b3cba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.265052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50902d6-6b9c-4a5e-b166-b3f34a4b3cba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.265516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50902d6-6b9c-4a5e-b166-b3f34a4b3cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.306915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c50902d6-6b9c-4a5e-b166-b3f34a4b3cba None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:19.308355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c50902d6-6b9c-4a5e-b166-b3f34a4b3cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:19.370937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:19.390739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df65299-53d2-40f7-9825-ec0e0c105c1b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:19.401606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 322/631] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:19.428324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:19.439385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:19.440441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.440868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.441310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.442353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:19.442851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:19.444546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:19.472639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 310/632] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:43:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:19.473715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50902d6-6b9c-4a5e-b166-b3f34a4b3cba None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:19.596899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50902d6-6b9c-4a5e-b166-b3f34a4b3cba None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:19.610449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:19.640606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d564021-d049-4ab8-a8a7-e97d1c7db8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:19.641138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d564021-d049-4ab8-a8a7-e97d1c7db8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.641415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d564021-d049-4ab8-a8a7-e97d1c7db8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.641684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d564021-d049-4ab8-a8a7-e97d1c7db8f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.741259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:19.767364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d564021-d049-4ab8-a8a7-e97d1c7db8f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:19.768422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71ba98-081b-4008-ab70-e5d356d75481 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:19.768866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d564021-d049-4ab8-a8a7-e97d1c7db8f7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:19.804907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:19.805535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.805815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.806089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.816776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:19.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:19.823398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:19.824230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 311/633] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:19.824937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:19.874020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:19.884334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 323/634] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:19.888871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1da1d-bfe6-41d4-91e6-0a5ed2899f98 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.888871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1da1d-bfe6-41d4-91e6-0a5ed2899f98 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.888871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1da1d-bfe6-41d4-91e6-0a5ed2899f98 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.888871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 324/635] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:19.888871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1da1d-bfe6-41d4-91e6-0a5ed2899f98 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:19.933843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4e335-f75f-4458-b499-b7211ee49c68 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:19.968253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:19.968892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:19.969163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:19.969430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:19.975949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:19.980694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:19.985482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:19.986639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:19.987451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 312/636] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:19.988135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.003464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1da1d-bfe6-41d4-91e6-0a5ed2899f98 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.040200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d4a00-2ec6-4cbb-b2d6-d2731441f8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.041085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d4a00-2ec6-4cbb-b2d6-d2731441f8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.041369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d4a00-2ec6-4cbb-b2d6-d2731441f8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.041642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d4a00-2ec6-4cbb-b2d6-d2731441f8b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.042303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d4a00-2ec6-4cbb-b2d6-d2731441f8b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:20.042604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d4a00-2ec6-4cbb-b2d6-d2731441f8b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.044452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d4a00-2ec6-4cbb-b2d6-d2731441f8b5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.052664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 325/637] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:20.052664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe888725-e914-499f-b5cd-5737a76bef3e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.052664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe888725-e914-499f-b5cd-5737a76bef3e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.052664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe888725-e914-499f-b5cd-5737a76bef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.073645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fe888725-e914-499f-b5cd-5737a76bef3e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:20.074822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fe888725-e914-499f-b5cd-5737a76bef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:20.097268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e593cc2-b055-481c-8a21-85f93d0f9017 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.107109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:20.138308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.139019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.139365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.139670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.144411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 326/638] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:20.145423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe888725-e914-499f-b5cd-5737a76bef3e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.149720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.151136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:20.165648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:20.166678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:20.167546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:20.167546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:20.167546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:20.171235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 313/639] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:20.172057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.237901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:20.279882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe888725-e914-499f-b5cd-5737a76bef3e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.307439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711696bd-78d4-43c7-9d3e-9c04a33058ac None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.331597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.332813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.333312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.333963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.335250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:20.335982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.338321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.342815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.343551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.343893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.344255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.351129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 327/640] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:20.352379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.359559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.363755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:20.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:20.380518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:20.381410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:20.381410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:20.381410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:20.385661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 314/641] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:20.386495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.401778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:20.484487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e32d8-1582-44be-b79c-e53cec7087a8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.492510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e362e8-a6ff-4966-b41d-9bb751341716 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.523324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.523925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.524203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.524502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.525135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:20.525434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.526787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.528292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.528414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:20.529215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.529672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.530118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.540611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 328/642] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:20.541406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.546082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.547593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:20.562984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:20.563995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:20.564930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:20.564930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:20.564930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:20.564930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 315/643] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:20.565167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.627592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:20.650885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879086b-c5e0-4913-a071-81e3ff07f694 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.667821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6908d2-5c11-4b0c-8bf7-401a3f9ba337 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.686663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.687274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.687550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.687820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.688486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:20.688795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.690158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.702071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 329/644] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:20.703487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.704126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.704433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.704704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.707425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.716684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:20.722006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:20.722910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 316/645] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:20.725796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.772525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:20.819262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82cc1-0e25-4073-8e2c-ed9eee672a54 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.832527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f0cea-a600-4546-a829-387fde71fe6d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:20.856006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.856696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.856975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.857247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.858063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:20.865448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.866930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.869732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:20.870332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:20.870608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:20.870879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:20.871510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:20.871805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:20.873257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:20.878165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:20.879004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 330/646] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:20.880507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.894601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 317/647] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:20.894601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:20.987623 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ae0b8-fb2f-4051-9e10-5e21f04a8d90 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.000640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dbcb-facb-4f07-b98d-cc26499178d9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.001719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.023945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.024533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.024809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.025080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.032594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.034041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.036717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.037308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.037579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.038396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.039043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:21.039338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.040689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:21.046170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:21.047143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 331/648] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:21.048085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.054846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 318/649] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:21.057225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.110218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.167213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e743e03-003d-4c61-a61d-bc01c82e500f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.182917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d5a1d-a3b9-4d58-ac9a-82f704ac295c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.203680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.204332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.204610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.204880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.213391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.214801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.218330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.218930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.219208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.219481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.220165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:21.220503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.221917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:21.225583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:21.226649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 319/650] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:21.232386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.238420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 332/651] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:21.239278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.271501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.336346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c1d08-b0d4-4bd4-bc5c-6be6d5f45c3b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.337647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a670a8-2222-46ba-8e99-a07f7ff1472e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.344211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.370491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.370938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ba57a-1783-4161-b4c1-a23e8b027f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.371114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.371387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.371544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ba57a-1783-4161-b4c1-a23e8b027f40 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.371656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.371826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ba57a-1783-4161-b4c1-a23e8b027f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.372102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ba57a-1783-4161-b4c1-a23e8b027f40 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.372858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ba57a-1783-4161-b4c1-a23e8b027f40 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:21.373167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ba57a-1783-4161-b4c1-a23e8b027f40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.374641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ba57a-1783-4161-b4c1-a23e8b027f40 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.381903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.383321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.392385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 320/652] 10.222.0.50 () {56 vars in 1137 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:21.393567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d64d13-ba39-4585-9f5a-b5951740bfab None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.393852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d64d13-ba39-4585-9f5a-b5951740bfab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.394124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d64d13-ba39-4585-9f5a-b5951740bfab None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.395517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 321/653] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:21.397719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d64d13-ba39-4585-9f5a-b5951740bfab None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:21.398062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:21.398997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:21.399865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:21.399865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:21.399865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:21.399865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 333/654] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:21.399865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.485681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.501566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1792b3c-d6d2-4de7-a013-6a141196eee1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.513575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d64d13-ba39-4585-9f5a-b5951740bfab None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.536826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.537431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.537741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.538013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.538735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=95b365b4794c4edebe306e09336c42f3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:21.539048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.540524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.546186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 334/655] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/role_assignments?user.id=95b365b4794c4edebe306e09336c42f3&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:21.547386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.548817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46959806-3089-44a7-ab72-472854bf5d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.549478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959806-3089-44a7-ab72-472854bf5d9b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.549814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959806-3089-44a7-ab72-472854bf5d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.550135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959806-3089-44a7-ab72-472854bf5d9b None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.550933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959806-3089-44a7-ab72-472854bf5d9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:21.551296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959806-3089-44a7-ab72-472854bf5d9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.552838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959806-3089-44a7-ab72-472854bf5d9b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.586879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 322/656] 10.222.0.50 () {56 vars in 1137 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:21.586879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd3678-3483-4a63-aa0e-5ce390407a24 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.586879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd3678-3483-4a63-aa0e-5ce390407a24 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.586879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd3678-3483-4a63-aa0e-5ce390407a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.608184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a2cd3678-3483-4a63-aa0e-5ce390407a24 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:21.610023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a2cd3678-3483-4a63-aa0e-5ce390407a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:21.640688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.662793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55dc69-a2d0-46a3-bb23-df321a161276 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.703153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.703450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 323/657] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:21.704352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd3678-3483-4a63-aa0e-5ce390407a24 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.712950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.713231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.713509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.735332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.815524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.817055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.826984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd3678-3483-4a63-aa0e-5ce390407a24 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:21.865437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:21.866730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:21.867015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:21.867283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:21.867917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:21.868352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:21.869814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:21.882640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 324/658] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:21.883374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.898818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.933060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 335/659] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:21.933968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:21.984433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:21.993815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2befaff-a3b7-4daf-962c-b98d1ce673d9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.030457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.031069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.031348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.031621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.041423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8753eb-2b72-4dbf-ac1f-cfa187090cef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.076719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.077268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.077546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.077818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.084958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.086411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:22.098702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:22.101773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 336/660] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:22.102768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.106748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.108223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.123288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.207193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b602-d25b-4653-939f-418e167d1af0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.219999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 325/661] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:22.219999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.221215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.246121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.247101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.247460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.247818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.259069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.260665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:22.275967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:22.277113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:22.278009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:22.278009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:22.278009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:22.278009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 337/662] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:22.278009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.333666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06c21e-3722-473e-91d4-438fd1ba62a0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.347661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.378896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.379533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.379813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.380089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.389112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f89ca-7ea6-4ea7-81a5-015b0b6831a7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.391005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:22.395831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:22.396813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:22.398136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:22.402364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 326/663] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:22.402976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.425022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.425635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.425914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.426189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.426914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:22.427334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.428839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.454094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 338/664] 10.222.0.50 () {56 vars in 1139 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:22.455022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.477144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.518165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836b770-86e4-4b75-812c-1d884bfa3caf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.554586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.555217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.555495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.555825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.561003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bc784-8318-4cd5-a4a1-ab5ec1cab95e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.566442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.567879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:22.583338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:22.585075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:22.585969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:22.585969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:22.585969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:22.585969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 327/665] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:22.586206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.605285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.606374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.606814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.607351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.608445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:22.608904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.610427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.614451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.630429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 339/666] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:22.631312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.691755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f22ae-c6f5-415b-9b5e-f2e52a9c7933 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.716530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.728754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.729378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.729657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.729929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.730607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:22.730912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.732329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.739819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28301a9-6c2a-4a41-8590-b053e990b0ab None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.745937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 328/667] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:22.745937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.785580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.786591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.786938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.787471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.835479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.852473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eeacb7-abd1-4b8e-a80f-e3d16fed01dd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:22.855336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.856813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.888755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:22.889358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:22.889638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:22.889908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:22.890541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:22.890840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:22.892179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:22.904234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 329/668] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:22.905040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:22.945145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:22.966674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 340/669] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:22.967623 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.013950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d56c8-2673-4bef-9295-b102456e92e6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.048089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.048743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.049022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.049291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.056482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.058370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:23.070483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:23.071553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 330/670] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:23.072474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.080897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f48f-f52a-411f-816f-375a32bf4405 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.081832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.116439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.116990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.117269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.117536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.127974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.129518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:23.144742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:23.145889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:23.147083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:23.147083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:23.147083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:23.147656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 341/671] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:23.148455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.180648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f258561-f575-46d2-ba86-5e785f22b224 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.182135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.218323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.218949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.219225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.219503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.226222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.227645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:23.239491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:23.240400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:23.241810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 331/672] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:23.242819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.259815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb16900-8498-494d-9cc2-eeaeec494c57 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.299512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.300293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.301068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.301415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.313799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.316688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.318491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:23.334075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:23.335070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:23.335930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:23.335930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:23.335930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:23.335930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 342/673] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:23.335930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.355729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135285-8b20-4e72-a767-cc6832fc466c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.392500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.393029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.393301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.393660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.394254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:23.394559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.396077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.407972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 332/674] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:23.408840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.427924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.445640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f224e3f-83fc-4daa-9dbd-91eb50762f3c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.478512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.479112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.479393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.479669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.480331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:23.480634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.481990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.494570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 343/675] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:23.495381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.517039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4774ac-45ab-45d2-b014-6c16f7955bbf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.544211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.554734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.555340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.555615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.555883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.556601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:23.556901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.558307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.569906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 333/676] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:23.570582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.597140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80890e-abd3-4b19-9d92-a5dd13e79278 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.630888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.631519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.631795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.632067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.632740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:23.633046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.634405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.645152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 344/677] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:23.646220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.649771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.670908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbc19-3bcd-4df0-8793-5a39ede39287 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.704102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7672f3-4862-4ebe-b152-82d35e02ce80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.704836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7672f3-4862-4ebe-b152-82d35e02ce80 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.705112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7672f3-4862-4ebe-b152-82d35e02ce80 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.705378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7672f3-4862-4ebe-b152-82d35e02ce80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.706085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7672f3-4862-4ebe-b152-82d35e02ce80 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=b68fc66dbef44519a2212d35655bf6a7) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:23.706387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7672f3-4862-4ebe-b152-82d35e02ce80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.707793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7672f3-4862-4ebe-b152-82d35e02ce80 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.713952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 334/678] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/role_assignments?user.id=b68fc66dbef44519a2212d35655bf6a7&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:23.715192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0ee98-b143-437f-9df3-6b2cae03c43c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.715547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0ee98-b143-437f-9df3-6b2cae03c43c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.715893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0ee98-b143-437f-9df3-6b2cae03c43c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.717206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 335/679] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:23.717996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0ee98-b143-437f-9df3-6b2cae03c43c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.761880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410498eb-42ca-4a7f-b447-695242cbeea5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.780828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.800828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.801470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.801749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.802021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.810551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.812015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:23.824648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:23.825675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 345/680] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:23.826627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:23.846597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0ee98-b143-437f-9df3-6b2cae03c43c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:23.887915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6c286-25b6-4cc5-83fa-c8c570bf0084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:23.888549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6c286-25b6-4cc5-83fa-c8c570bf0084 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.888826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6c286-25b6-4cc5-83fa-c8c570bf0084 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.889096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6c286-25b6-4cc5-83fa-c8c570bf0084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.890267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6c286-25b6-4cc5-83fa-c8c570bf0084 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=88273fe4807b4f08911298b7bf0c3653) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:23.890687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6c286-25b6-4cc5-83fa-c8c570bf0084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:23.893277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6c286-25b6-4cc5-83fa-c8c570bf0084 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:23.900357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 336/681] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/role_assignments?user.id=88273fe4807b4f08911298b7bf0c3653&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:23.901266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653909a4-76c3-4e6b-a047-904457873da9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:23.901546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653909a4-76c3-4e6b-a047-904457873da9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:23.901819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653909a4-76c3-4e6b-a047-904457873da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:23.921972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-653909a4-76c3-4e6b-a047-904457873da9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:23.923183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-653909a4-76c3-4e6b-a047-904457873da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:23.944780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:23.958500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c90405-2b3d-4e31-98e6-9fcc6dfc3e0f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.000463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 337/682] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:24.001381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653909a4-76c3-4e6b-a047-904457873da9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.020749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.021401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.023171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.023461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.041170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:24.042913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:24.065193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:24.066332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:24.067272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:24.072608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 346/683] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:24.076293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.083696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:24.155726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653909a4-76c3-4e6b-a047-904457873da9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.203424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.204040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.204345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.204620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.204810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa0331-97c9-491b-9e1c-93e7e7ba55cd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.205349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:24.205649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:24.207458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:24.230570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 338/684] 10.222.0.50 () {56 vars in 1135 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:24.233398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:24.236023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.246401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.247535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.247825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.248100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.248833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:24.249144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:24.251478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:24.270364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 347/685] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:24.279177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.363909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1d97e-24f0-436e-baad-e8c22ce0d34f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.365999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:24.404882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.406554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.407147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.407713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.413510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb2aae-2146-4a41-9a0f-1c7eec1e3dfb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.459659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.460399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.460755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.461106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.462390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=e1c3b7f7c83f48aca27fbd46202f6506) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:24.462757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:24.464756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:24.474098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 348/686] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/role_assignments?user.id=e1c3b7f7c83f48aca27fbd46202f6506&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:24.481172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.516927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:24.519544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:24.562433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:24.619298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50907da-7cd1-4f94-8240-9aa6f081e634 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.655944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.656761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.657042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.657400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.671735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:24.681112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:24.682170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:24.683257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:24.686226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 349/687] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:24.695377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.737249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 339/688] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 01:43:24.745731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.747911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:24.812486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4371-ee72-4707-81cf-aa11a191be87 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.822658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:24.861599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.862219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.862498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.862772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.863444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:24.863747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:24.865170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:24.879069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435887c-ab7e-453d-a57c-51309662a522 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:24.885557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 350/689] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:24.891507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:24.929700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:24.930618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:24.931060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:24.931481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:24.942999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:24.944846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:24.970790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:24.971967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:24.973108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:24.973108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:24.973108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:24.978951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 340/690] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:24.985929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.070043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc90d4-f5c0-4e5c-a3cf-ba97e25ee566 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.084741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:25.106830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.107689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.108158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.108681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:25.120908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:25.122685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:25.137538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:25.138728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:25.139802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:25.139802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:25.139802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:25.141559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 351/691] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:25.151311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.158089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db70f5-632b-4bc7-9074-8c7cc43b290b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.193522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.194087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.194363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.194632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:25.195270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:25.195566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:25.198204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:25.203829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:25.210916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 341/692] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:25.216847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.364196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795841f-b954-442c-9d00-4647a83bb3c1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.368592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3bbfc-5543-47db-ba1e-322bb4456343 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.393997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:25.426625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.427235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.427513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.427783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:25.428482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:25.428784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:25.430728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:25.443690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 352/693] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:25.444933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.445587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.445876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.446153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:25.449149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.461289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:25.462738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:25.486052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:25.487197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:25.488066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:25.495836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 342/694] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:25.501390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.542807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:25.564712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9872c-3303-4094-8e24-2b5f64151974 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.603557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.604202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.604501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.604773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:25.613314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:25.614821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:25.628166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:25.630763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:25.633519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 353/695] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:43:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:25.640368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.751813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441f05f-de3b-4ea0-8b3d-464928e18e55 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.790894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.791749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.792581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.793020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:25.794038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:25.794602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:25.796483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:25.812773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:25.818310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 354/696] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:43:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:25.818472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91b039-3c79-4880-b909-5bd85acd99f8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.825787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.927662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.928994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.929339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.929679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:25.930442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:25.930811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:25.932344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:25.942725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87c977-9a83-4dd8-bbe2-4f8c988b1061 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:25.959848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 343/697] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:25.965199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:25.987222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:25.988652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:25.989111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:25.989741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.020742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:26.081209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:26.086116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.088063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.088182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7594068-4008-4898-8d5f-f071c2f89a59 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:26.130723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff0caf-b5c3-4cd7-a3ff-1ed33d3048cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:26.131521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff0caf-b5c3-4cd7-a3ff-1ed33d3048cd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.131860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff0caf-b5c3-4cd7-a3ff-1ed33d3048cd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.132200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff0caf-b5c3-4cd7-a3ff-1ed33d3048cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.133012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff0caf-b5c3-4cd7-a3ff-1ed33d3048cd None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=5956fdd8d6b54640970cde2705a83260) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:26.133380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff0caf-b5c3-4cd7-a3ff-1ed33d3048cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.134875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff0caf-b5c3-4cd7-a3ff-1ed33d3048cd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.141428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 344/698] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:25 2022] GET /identity/v3/role_assignments?user.id=5956fdd8d6b54640970cde2705a83260&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:26.142679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f02429-a40d-49ae-9003-284efa5588db None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.143389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f02429-a40d-49ae-9003-284efa5588db None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.143389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f02429-a40d-49ae-9003-284efa5588db None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.148339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 345/699] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:26.151581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c317a3-dbec-4f85-b591-5e1bff254754 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.151581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c317a3-dbec-4f85-b591-5e1bff254754 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.151891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c317a3-dbec-4f85-b591-5e1bff254754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.175300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-79c317a3-dbec-4f85-b591-5e1bff254754 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:26.176888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-79c317a3-dbec-4f85-b591-5e1bff254754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:26.207614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 355/700] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:26.216291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:26.283865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 346/701] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:26.285234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601483bb-3953-4882-952b-c34fb402f168 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.285520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601483bb-3953-4882-952b-c34fb402f168 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.285795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601483bb-3953-4882-952b-c34fb402f168 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.288886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 347/702] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:26.293583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc5b49-94d0-4e95-8b5e-e31f58d465b7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.293842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc5b49-94d0-4e95-8b5e-e31f58d465b7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.294173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc5b49-94d0-4e95-8b5e-e31f58d465b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.313241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-38fc5b49-94d0-4e95-8b5e-e31f58d465b7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:26.314623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-38fc5b49-94d0-4e95-8b5e-e31f58d465b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:26.341240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:26.342385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:26.351586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecfed5-90ee-4b83-a40e-4c3d1dd5745f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:26.410230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:26.411108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.411563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.411930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.420203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 348/703] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:26.422751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.424325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:26.439186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:26.440387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:26.441460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:26.441460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:26.441460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:26.442753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 356/704] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:26.448819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc5b49-94d0-4e95-8b5e-e31f58d465b7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:26.558604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc5b49-94d0-4e95-8b5e-e31f58d465b7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:26.592676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:26.593763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:26.594376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.594650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.594918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.595557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:26.595852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.597276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.617045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 349/705] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:26.623171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:26.644791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:26.721163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7926b8d-912b-4b4b-8179-0d14273150b9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:26.737196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8298e6a-3238-4903-a984-3e050a2df4e9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:26.764092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:26.764782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.765100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.765354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.765937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.767403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.780360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:26.780360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.780360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:26.780678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:26.781613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:26.782612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:26.783470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.783568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 357/706] 10.222.0.50 () {56 vars in 1129 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:26.783568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:26.791743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.793432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.808689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:26.808849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:26.809970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:26.811188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:26.813007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 350/707] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:26.817837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:26.894972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:26.902257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc359a31-5c8b-47cc-8150-83eda4dd1569 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:26.916576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19f795-26a6-4883-8fef-7cc2740cd164 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:26.938071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934f87-51d3-476d-8cfe-f6505ab6616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:26.938822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934f87-51d3-476d-8cfe-f6505ab6616d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.939105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934f87-51d3-476d-8cfe-f6505ab6616d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.939478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934f87-51d3-476d-8cfe-f6505ab6616d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.950687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934f87-51d3-476d-8cfe-f6505ab6616d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.950825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:26.951343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.951619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.951889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.952155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934f87-51d3-476d-8cfe-f6505ab6616d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.952557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:26.952860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:26.954193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:26.963141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 351/708] 10.222.0.50 () {56 vars in 1124 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:26.963141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0b934f87-51d3-476d-8cfe-f6505ab6616d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:26.972337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:26.973278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:26.974152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:26.974152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:26.974152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:26.974152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 358/709] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:26.974793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421be594-3b38-4a15-b2d2-809e98408211 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:26.975020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421be594-3b38-4a15-b2d2-809e98408211 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:26.975303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421be594-3b38-4a15-b2d2-809e98408211 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:26.980518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 359/710] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:26.981292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421be594-3b38-4a15-b2d2-809e98408211 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:27.055996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:27.071676 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad6f63-7d56-4ba4-91d6-57cd68d53f3f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:27.094961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421be594-3b38-4a15-b2d2-809e98408211 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:27.108149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:27.108863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.109149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.109435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.110114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:27.110423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:27.112102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:27.129639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 352/711] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:27.129639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:27.129639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:27.136423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61077f-a46d-4be0-8d86-1180a32b1b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:27.137035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61077f-a46d-4be0-8d86-1180a32b1b53 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.137313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61077f-a46d-4be0-8d86-1180a32b1b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.137643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61077f-a46d-4be0-8d86-1180a32b1b53 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.138337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61077f-a46d-4be0-8d86-1180a32b1b53 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:27.138647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61077f-a46d-4be0-8d86-1180a32b1b53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:27.140027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61077f-a46d-4be0-8d86-1180a32b1b53 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:27.146129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 360/712] 10.222.0.50 () {56 vars in 1124 bytes} [Tue Aug 30 01:43:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:27.147408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406ad7-d8f4-4de3-8cb6-e15c3d3b6fe0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.147692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406ad7-d8f4-4de3-8cb6-e15c3d3b6fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.147966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406ad7-d8f4-4de3-8cb6-e15c3d3b6fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.170160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c4406ad7-d8f4-4de3-8cb6-e15c3d3b6fe0 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:27.172352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c4406ad7-d8f4-4de3-8cb6-e15c3d3b6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:27.264271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 361/713] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:27.264271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406ad7-d8f4-4de3-8cb6-e15c3d3b6fe0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:27.291382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2046d87-484f-4adf-87f9-7a8710c5e587 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:27.342503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:27.343315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.343596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.343872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.344971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:27.345273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:27.346925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:27.351886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:27.368127 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 353/714] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:27.371424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:27.401087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406ad7-d8f4-4de3-8cb6-e15c3d3b6fe0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:27.441582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:27.442213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.442474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.442748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.444838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=95b365b4794c4edebe306e09336c42f3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:27.445169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:27.446604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:27.457519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 362/715] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:27 2022] GET /identity/v3/role_assignments?user.id=95b365b4794c4edebe306e09336c42f3&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:27.490117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:27.542118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2b50b-33cb-4b6a-8846-e781f1107ddc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:27.590454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:27.591091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.591369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.591641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.592351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:27.593874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:27.619693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:27.627231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 354/716] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:43:27 2022] POST /identity/v3/users => generated 254 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:27.749757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dd58-36cb-45d9-b3ce-d9758acb1a58 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:27.791475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:27.806416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:27.807446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.807927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.808445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.809259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:27.810882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:27.821307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:27.822363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:27.823254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:27.830555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 363/717] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:27 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:27.832467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:27.944785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cb04f-313b-42f3-9d4a-32507639fe7f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:27.963103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:27.980044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f81e8d-e34d-4a25-9995-e125de0da4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:27.980687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f81e8d-e34d-4a25-9995-e125de0da4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:27.981416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f81e8d-e34d-4a25-9995-e125de0da4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:27.982055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f81e8d-e34d-4a25-9995-e125de0da4d8 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:27.982742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f81e8d-e34d-4a25-9995-e125de0da4d8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:27.983586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f81e8d-e34d-4a25-9995-e125de0da4d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:27.984987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f81e8d-e34d-4a25-9995-e125de0da4d8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:27.989977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 355/718] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:28.000632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:28.120393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994930b-fb1f-4fac-a642-da2ca913b4c0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:28.157387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe714a4-c925-4960-b4e8-e023b43443d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:28.158381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe714a4-c925-4960-b4e8-e023b43443d3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:28.158863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe714a4-c925-4960-b4e8-e023b43443d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:28.159359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe714a4-c925-4960-b4e8-e023b43443d3 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:28.160196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe714a4-c925-4960-b4e8-e023b43443d3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:28.160728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe714a4-c925-4960-b4e8-e023b43443d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:28.162361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe714a4-c925-4960-b4e8-e023b43443d3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:28.171279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 364/719] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:28.853967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd3b8-9ae9-4dcd-a35b-c4b665f6ef9a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:28.854502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd3b8-9ae9-4dcd-a35b-c4b665f6ef9a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:28.854502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dd3b8-9ae9-4dcd-a35b-c4b665f6ef9a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:28.856094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 356/720] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:28.861987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444b460-0379-4297-8a34-bbcbae51dfd9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:28.862782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444b460-0379-4297-8a34-bbcbae51dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:28.863257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444b460-0379-4297-8a34-bbcbae51dfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:28.889229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4444b460-0379-4297-8a34-bbcbae51dfd9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:28.890426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4444b460-0379-4297-8a34-bbcbae51dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:28.960251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:28.983153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 365/721] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:29.336226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd3b8-9ae9-4dcd-a35b-c4b665f6ef9a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:29.384207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9dcd-8ff2-46bd-a32a-e9987090be92 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:29.384535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9dcd-8ff2-46bd-a32a-e9987090be92 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:29.384805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9dcd-8ff2-46bd-a32a-e9987090be92 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:29.386411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 366/722] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:29.391188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f0984-f7b8-429e-9153-2079e21db213 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:29.391477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f0984-f7b8-429e-9153-2079e21db213 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:29.391754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f0984-f7b8-429e-9153-2079e21db213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:29.415560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9c6f0984-f7b8-429e-9153-2079e21db213 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:29.416780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9c6f0984-f7b8-429e-9153-2079e21db213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:29.488478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dd3b8-9ae9-4dcd-a35b-c4b665f6ef9a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:29.523559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:29.524198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:29.524502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:29.524781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:29.535036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:29.540396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:29.541308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:29.542147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 357/723] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:29.550316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:29.550967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 367/724] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:43:29.628653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:29.700951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748073d9-3bc4-4d57-b6e9-9f6bc780ff25 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:29.743729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:29.744370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:29.744652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:29.744928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:29.745589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:29.745892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:29.747256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:29.760740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 358/725] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:29.766221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f0984-f7b8-429e-9153-2079e21db213 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:29.901591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f0984-f7b8-429e-9153-2079e21db213 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:29.924608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:29.936766 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf13413-6b60-401a-bfb5-31540737b774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:29.937401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13413-6b60-401a-bfb5-31540737b774 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:29.937679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13413-6b60-401a-bfb5-31540737b774 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:29.937953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13413-6b60-401a-bfb5-31540737b774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:29.948997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf13413-6b60-401a-bfb5-31540737b774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:29.950459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf13413-6b60-401a-bfb5-31540737b774 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-bcf13413-6b60-401a-bfb5-31540737b774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:29.965483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:29.966427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:29.967291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:29.967291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:29.967291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:29.971276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 368/726] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:29.972736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:30.042107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9386dd-c3ce-4ac5-8501-a54d589f3306 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.042476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9386dd-c3ce-4ac5-8501-a54d589f3306 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.042658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9386dd-c3ce-4ac5-8501-a54d589f3306 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.044369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 369/727] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:30.045067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9386dd-c3ce-4ac5-8501-a54d589f3306 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:30.103406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71107-8b80-44d4-9745-4ad401f9ff42 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:30.140643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093003b2-23bb-494c-b6a0-d483cc422f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:30.141723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093003b2-23bb-494c-b6a0-d483cc422f49 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.142208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093003b2-23bb-494c-b6a0-d483cc422f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.142790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093003b2-23bb-494c-b6a0-d483cc422f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.143631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093003b2-23bb-494c-b6a0-d483cc422f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:30.144137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093003b2-23bb-494c-b6a0-d483cc422f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:30.146035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093003b2-23bb-494c-b6a0-d483cc422f49 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:30.152728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9386dd-c3ce-4ac5-8501-a54d589f3306 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:30.160917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 359/728] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:30.164469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:30.164978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd923ce-01b9-49a7-bdba-220ab44b4306 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.165481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd923ce-01b9-49a7-bdba-220ab44b4306 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.165983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd923ce-01b9-49a7-bdba-220ab44b4306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.185730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8fd923ce-01b9-49a7-bdba-220ab44b4306 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:30.187423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8fd923ce-01b9-49a7-bdba-220ab44b4306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:30.194910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:30.195585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.195865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.196136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.196836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:30.198314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:30.228303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 370/729] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:43:30 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:30.228303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:30.278182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:30.305153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 360/730] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:30.306673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7af68-83be-49e9-a277-c0a8eefb860a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.306960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7af68-83be-49e9-a277-c0a8eefb860a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.307258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7af68-83be-49e9-a277-c0a8eefb860a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.308879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 361/731] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:30.313997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af5ea7-d11c-40f2-b5e1-0e77a68327b4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.314524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af5ea7-d11c-40f2-b5e1-0e77a68327b4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.314952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af5ea7-d11c-40f2-b5e1-0e77a68327b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.339516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f8af5ea7-d11c-40f2-b5e1-0e77a68327b4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:30.340935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f8af5ea7-d11c-40f2-b5e1-0e77a68327b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:30.372118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86581a41-0869-46ad-8ce9-49e83444c88a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:30.413861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27873a46-840c-4046-bee6-a49396764c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:30.414593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27873a46-840c-4046-bee6-a49396764c5a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.414967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27873a46-840c-4046-bee6-a49396764c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.415348 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27873a46-840c-4046-bee6-a49396764c5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.416320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 362/732] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 01:43:30.425587 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27873a46-840c-4046-bee6-a49396764c5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:30.427317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27873a46-840c-4046-bee6-a49396764c5a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-27873a46-840c-4046-bee6-a49396764c5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:30.441814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:30.443948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:30.447063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:30.452293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 371/733] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:30.453509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af5ea7-d11c-40f2-b5e1-0e77a68327b4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:30.480675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:30.522883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad25eb-9eb8-4dd6-96bd-62f6f351757b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.523163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad25eb-9eb8-4dd6-96bd-62f6f351757b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.523434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad25eb-9eb8-4dd6-96bd-62f6f351757b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.524940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 372/734] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:30.529778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffba6-1ead-4174-b632-6662a0682a09 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.530067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffba6-1ead-4174-b632-6662a0682a09 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.530340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffba6-1ead-4174-b632-6662a0682a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.553683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d33ffba6-1ead-4174-b632-6662a0682a09 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:30.554878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d33ffba6-1ead-4174-b632-6662a0682a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:30.563406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af5ea7-d11c-40f2-b5e1-0e77a68327b4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:30.602298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:30.602906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.603182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.603453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.604344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:30.604632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:30.606130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:30.628732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 363/735] 10.222.0.50 () {56 vars in 1135 bytes} [Tue Aug 30 01:43:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:30.632123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:30.635278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 373/736] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 01:43:30.649618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ffba6-1ead-4174-b632-6662a0682a09 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:30.673297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:30.737821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:30.762049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ffba6-1ead-4174-b632-6662a0682a09 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:30.766834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71dae9-e71f-4e7d-9d19-0542f0bdc469 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:30.809371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:30.810007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.810286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.810558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.816469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:30.817547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.818489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.818489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.823076 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:30.828024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:30.829309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:30.830163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 374/737] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:30.830920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:30.914079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:30.915565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:30.916292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:30.941671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740f204-245f-44b2-bd93-30a95c81a102 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:30.978667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:30.979301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:30.979582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:30.979857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:30.990916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:30.994431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:30.997346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:30.998243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:30.999115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 375/738] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:30.999859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.022952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 364/739] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 01:43:31.025285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.109896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742a01f-7790-49f4-bb67-eed65a5dd7b8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.131039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f35d-3b77-496b-a72f-3e1fb0b71259 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.145340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.145943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.146221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.146705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.147208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:31.147511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.148916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.152048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:31.159502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 376/740] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:31.160363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.170745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.171349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.171853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.172038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.172680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.174032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.192356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 365/741] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:43:31 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:31.205297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.272182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:31.279752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23fbf-f269-4b7c-b435-56d431a36576 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.332327 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.333060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.333341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.333611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.334341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:31.334645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.336234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.346339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 377/742] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:31.347184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.356230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac3468-b895-459c-9792-5256e1fcc846 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.396985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.397663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.397942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.398214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.411649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.413162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.421695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:31.430973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:31.431924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:31.432905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:31.432905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:31.432905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:31.432905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 366/743] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:31.433546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.474352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b2ed4-7e1b-4840-9241-3a246fa09b80 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.509024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.509656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.509933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.510210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.520929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.522517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:31.537011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:31.538529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:31.539427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:31.539427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:31.539427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:31.539427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 378/744] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:31.540090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.543527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:31.557051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a3b1f-7d11-4008-a1d1-d25b437b902c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.597970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.598597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.598870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.599138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.610997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.612713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:31.627815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:31.629026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:31.629906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:31.629906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:31.629906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:31.629906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 367/745] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:31.629906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.688091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97937d-ef4e-42e9-8d3f-37cb002aa6c5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.697904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:31.725501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.726352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.726769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.727280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.728067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:31.728543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.730055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.744132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 379/746] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:31.745394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.752086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932ff98-9d0d-4cb4-bcad-eb621889d706 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.789378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.790583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.790583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.790583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.791138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:31.791439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.792828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.810014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 368/747] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:31.811265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.813817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:31.866483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf3fdd-c83d-4cbe-b374-abc4078af72e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.906336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.907378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.907867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.908480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.909431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:31.909929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.911600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.914957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c15b-4678-4efd-aeb3-cd52fccdb52b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:31.929872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 380/748] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:31.931476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:31.951373 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:31.952011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:31.952324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:31.952600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:31.959289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:31.961111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:31.964484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:31.972486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:31.973368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:31.974268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:31.978379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 369/749] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:31.979155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.041281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:32.051661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece9253-7946-412e-8504-d66ea43cf6a8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.081323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a866ed-41dd-416e-86ea-2d272c2cc535 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.091562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.092672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.093150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.093615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.102044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.103842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:32.117078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:32.118029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:32.118951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:32.119467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.119556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.120145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.120145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.121486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 381/750] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:32.122290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.128026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.129510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:32.141960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:32.142931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:32.143876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:32.147726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 370/751] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:43:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:32.148600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.231562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:32.247380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067fc4a-5c1a-443c-a6d4-a65f3983c658 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.251369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e51a8-876f-48be-8522-31d52b03de18 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.275098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:32.294437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.295407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.296294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.297120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.297706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.298103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.298434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.298748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.299551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:32.300333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:32.300699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.300987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.302310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.302755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.317303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 382/752] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:32.318282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.323257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 371/753] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:32.324434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.433309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5975-c2c7-4146-9a0c-e3bd921b6d9b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.436890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9990c5f-3956-4cf1-9371-0c92b6252df6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.473635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.474488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.475495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.476007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.477024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=5956fdd8d6b54640970cde2705a83260) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:32.477455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.477635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.478243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.478626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.478990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.479663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.479909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:32.480328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.482002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.486763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 372/754] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/role_assignments?user.id=5956fdd8d6b54640970cde2705a83260&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:32.487796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.502944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 383/755] 10.222.0.50 () {56 vars in 1149 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:32.513147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.527403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:32.546601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:32.616609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b3083-7976-4745-8eb8-e8f9b7a6b642 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.661733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce4fa5-5e63-4a8d-9292-d89ecd3de2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.663192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4fa5-5e63-4a8d-9292-d89ecd3de2ab None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.663649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4fa5-5e63-4a8d-9292-d89ecd3de2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.664080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce4fa5-5e63-4a8d-9292-d89ecd3de2ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.671578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80391f3a-1bfb-4c74-839c-dcc50f2565ae None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.676120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce4fa5-5e63-4a8d-9292-d89ecd3de2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.677774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce4fa5-5e63-4a8d-9292-d89ecd3de2ab None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2cce4fa5-5e63-4a8d-9292-d89ecd3de2ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:32.693503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:32.694661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:32.695773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:32.695773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:32.695773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:32.697536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 373/756] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:32.698870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13239418-8ff8-4e38-93df-ab54dd2dd433 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.699305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13239418-8ff8-4e38-93df-ab54dd2dd433 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.699711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13239418-8ff8-4e38-93df-ab54dd2dd433 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.705348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 374/757] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:32.706393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13239418-8ff8-4e38-93df-ab54dd2dd433 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.722117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.722731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.723009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.723281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.813116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:32.821493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.822602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13239418-8ff8-4e38-93df-ab54dd2dd433 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:32.823799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.839248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:32.864465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7005d8-49e0-46d8-b9b8-0fbcb0e0fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:32.865217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7005d8-49e0-46d8-b9b8-0fbcb0e0fd49 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.865733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7005d8-49e0-46d8-b9b8-0fbcb0e0fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.866110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7005d8-49e0-46d8-b9b8-0fbcb0e0fd49 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.867010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7005d8-49e0-46d8-b9b8-0fbcb0e0fd49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:32.867351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7005d8-49e0-46d8-b9b8-0fbcb0e0fd49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:32.868891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7005d8-49e0-46d8-b9b8-0fbcb0e0fd49 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:32.886307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 375/758] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:32.886307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23320-67bc-4679-97b3-0c9b7539c09c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:32.886307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23320-67bc-4679-97b3-0c9b7539c09c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:32.886307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23320-67bc-4679-97b3-0c9b7539c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:32.903604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-50e23320-67bc-4679-97b3-0c9b7539c09c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:32.904965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-50e23320-67bc-4679-97b3-0c9b7539c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:32.959414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 384/759] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:43:32.960595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:32.995622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 376/760] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:32.997027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23320-67bc-4679-97b3-0c9b7539c09c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.094925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:33.115147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23320-67bc-4679-97b3-0c9b7539c09c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.151185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f16c1-62b8-4ac9-b907-aaf57800e4f6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.155739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.157467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.157900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.158331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.163492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:33.174909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.176664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:33.194437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:33.195405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:33.196302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:33.196302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:33.196302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:33.199239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 377/761] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:33.204341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.226027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a945fc-04d2-46f6-aca1-5c917ca5d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.226698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a945fc-04d2-46f6-aca1-5c917ca5d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.226974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a945fc-04d2-46f6-aca1-5c917ca5d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.227247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a945fc-04d2-46f6-aca1-5c917ca5d4d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.249310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a945fc-04d2-46f6-aca1-5c917ca5d4d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.251519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a945fc-04d2-46f6-aca1-5c917ca5d4d3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-82a945fc-04d2-46f6-aca1-5c917ca5d4d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:33.274175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:33.275081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:33.275951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:33.284631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 385/762] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:33.286009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b931-8019-4233-afa4-3b7abf352707 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.286232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b931-8019-4233-afa4-3b7abf352707 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.286506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b931-8019-4233-afa4-3b7abf352707 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.288600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 386/763] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:33.294315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2fa6-fcdd-4a30-9863-2294169de146 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.294604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2fa6-fcdd-4a30-9863-2294169de146 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.294890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2fa6-fcdd-4a30-9863-2294169de146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.322323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b52b2fa6-fcdd-4a30-9863-2294169de146 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:33.324079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b52b2fa6-fcdd-4a30-9863-2294169de146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:33.325016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aefd64-9dad-4f53-800a-24ef61d3aaa7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.370225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.370754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.371026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.371315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.371918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:33.372214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.374103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.384985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:33.396541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 378/764] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:33.397778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.408501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 387/765] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:33.409264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b2fa6-fcdd-4a30-9863-2294169de146 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.503891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c8d3-c125-4389-98dc-78e3e26b6a4d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.536593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:33.541176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.541789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.542070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.542347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.543038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:33.543347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.544894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.545361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b2fa6-fcdd-4a30-9863-2294169de146 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.557302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 379/766] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:33.558013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.584992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.586116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.586637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.587144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.587925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.589739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:33.599265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:33.600968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:33.602014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:33.604719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 388/767] 10.222.0.50 () {56 vars in 1129 bytes} [Tue Aug 30 01:43:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:33.605605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.629408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:33.671531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e3157-bda3-4334-87fa-7146f77b92e5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.711149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.711800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.712322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.712460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.715225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817739e-6f87-4e4b-ac2a-6db98c6da100 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.719046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.720485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:33.732143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:33.733038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:33.733925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 380/768] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:33.734414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.756973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.757629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.757907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.758265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/95b365b4794c4edebe306e09336c42f3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.760195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=95b365b4794c4edebe306e09336c42f3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.761679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.775565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 389/769] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:33 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/95b365b4794c4edebe306e09336c42f3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:33.776808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.815607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:33.853480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e321e5-c938-47f0-9627-06486dd44e02 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.874952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:33.895938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.896498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.897461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.897868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:33.898427 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:33.898739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:33.900152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:33.905264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 381/770] 10.222.0.50 () {56 vars in 1124 bytes} [Tue Aug 30 01:43:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:33.906552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:33.945293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcac121-286c-47fe-994e-d99b4312afd2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:33.998041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:33.998705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:33.998955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:33.999205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] PATH_INFO: `/v3/services/network` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.000374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.001762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:34.012937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:34.013937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:34.015444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 390/771] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:33 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:34.016235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.035595 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204836f-7a6f-4640-861d-422a674bbc20 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.072877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.073567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.073986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.074267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.075012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:34.075324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.076905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.090149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 382/772] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.091143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.127163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:34.140581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:34.146270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432430a-54e9-431e-ba22-fdf4eef9fafd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.181761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.182368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.182643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.182911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.183535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:34.183832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.185232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.190559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 391/773] 10.222.0.50 () {56 vars in 1124 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.191514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.206761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4707-9eca-40ec-b324-8c5fe8fb7147 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.246057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.246672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.246953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.247229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.247860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] RBAC: Adding query filter params (name=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:34.248162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.249781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.255326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 383/774] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.256356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.295542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676f752-62be-4801-9bd1-6a8c64a2d0bb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.330548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.331154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.331426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.331695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.332440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=b5ef7dccfc5e43be993e201a19730a1a) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:34.332794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.334788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.340779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 392/775] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/role_assignments?user.id=b5ef7dccfc5e43be993e201a19730a1a&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.341575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.359443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:34.370366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477a868-5762-40df-9b45-5a438648dc65 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.379450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:34.411232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.411771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.412156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.413056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.413877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] RBAC: Adding query filter params (interface=public, service_id=0680f76bcfe34f31b1342bbf58eb0cc9, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:34.414187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.415651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.424333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 384/776] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/endpoints?service_id=0680f76bcfe34f31b1342bbf58eb0cc9&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.437480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.460718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abe150-7da4-4f58-9425-6274a05ef406 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.514945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.515625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.515936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.516168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.516824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] RBAC: Adding query filter params (type=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:34.517125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.518539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.526823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 393/777] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.536950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.629488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574d23d-f3c0-4af6-894b-9a2404048c44 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.649276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:34.671915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc034a2e-bdb1-40a2-bcfd-e41d7153c267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.672697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc034a2e-bdb1-40a2-bcfd-e41d7153c267 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.673374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc034a2e-bdb1-40a2-bcfd-e41d7153c267 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.673708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc034a2e-bdb1-40a2-bcfd-e41d7153c267 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.674570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc034a2e-bdb1-40a2-bcfd-e41d7153c267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.675923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc034a2e-bdb1-40a2-bcfd-e41d7153c267 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.676081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418cb1-2abf-4a4a-a2f4-6ed7e2e4d973 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.682999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 385/778] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/services => generated 772 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.684812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b13b30-747f-4ca3-be51-f9a5d5a394fe None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.685097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b13b30-747f-4ca3-be51-f9a5d5a394fe None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.685374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b13b30-747f-4ca3-be51-f9a5d5a394fe None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.687406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 386/779] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:34.693263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4e744-efb6-447a-82fa-4f97f5a94ca5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.694186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4e744-efb6-447a-82fa-4f97f5a94ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.700276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4e744-efb6-447a-82fa-4f97f5a94ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.722837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.723470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.723747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.728273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.728273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] RBAC: Adding query filter params (interface=public, service_id=693a626e129542d18827c37d246d759b, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:34.728273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.728273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.735940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 394/780] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/endpoints?service_id=693a626e129542d18827c37d246d759b&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:34.736692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bfe4e744-efb6-447a-82fa-4f97f5a94ca5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:34.738159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bfe4e744-efb6-447a-82fa-4f97f5a94ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:34.745421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:34.774675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:34.829291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 387/781] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:34.919605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501429f-b666-490c-99bb-21f1c51b8e96 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:34.957479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:34.958458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:34.958937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:34.959478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:34.960318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:34.961884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:34.967770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 395/782] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:43:34 2022] GET /identity/v3/services => generated 772 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:35.163317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4e744-efb6-447a-82fa-4f97f5a94ca5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:35.265607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4e744-efb6-447a-82fa-4f97f5a94ca5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:35.305652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:35.306303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:35.306583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:35.306856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:35.319049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:35.324291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:35.325189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:35.325997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:35.328577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 388/783] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:35.334066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:35.344891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:35.481421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd7c04-cb56-4734-9f95-61f72e96ade5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:35.523492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:35.524476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:35.524980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:35.525475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:35.526344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:35.526847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:35.528449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:35.538332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 396/784] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:35.543921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:35.575965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:35.660677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a350c1-aad0-4090-83b6-5ab28c0921d8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:35.696788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:35.697417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:35.697698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:35.697974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:35.708352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:35.710407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:35.726838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:35.727807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:35.728730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:35.728730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:35.728730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:35.728730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 389/785] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:35.733743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:35.848817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:35.893998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afd85-fe57-4d75-8c5a-f1478e4bea2c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:35.942529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e838ce3-f067-4fc2-829b-cb2ee00477a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:35.943135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e838ce3-f067-4fc2-829b-cb2ee00477a0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:35.943407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e838ce3-f067-4fc2-829b-cb2ee00477a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:35.943713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e838ce3-f067-4fc2-829b-cb2ee00477a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:35.944384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e838ce3-f067-4fc2-829b-cb2ee00477a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:35.944680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e838ce3-f067-4fc2-829b-cb2ee00477a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:35.946056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e838ce3-f067-4fc2-829b-cb2ee00477a0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:35.964176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 397/786] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:35.969672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:36.002819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e0268-6f20-450b-98ec-e299c3ffd40d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.003157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e0268-6f20-450b-98ec-e299c3ffd40d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.003381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e0268-6f20-450b-98ec-e299c3ffd40d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.005106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 398/787] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:36.010141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c27e85-232b-4cf5-9cf2-3f6404ee0ca6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.010416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c27e85-232b-4cf5-9cf2-3f6404ee0ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.010685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c27e85-232b-4cf5-9cf2-3f6404ee0ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.033370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-06c27e85-232b-4cf5-9cf2-3f6404ee0ca6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:36.034583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-06c27e85-232b-4cf5-9cf2-3f6404ee0ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:36.110862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:36.141283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 399/788] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:36.172017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c3b3d-7254-4456-9a2b-7f719efee90a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:36.233134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:36.234159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.234645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.235122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.247669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:36.249389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:36.263778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:36.264721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:36.265637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:36.268117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 390/789] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:36.274086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c27e85-232b-4cf5-9cf2-3f6404ee0ca6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:36.374066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:36.427002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c27e85-232b-4cf5-9cf2-3f6404ee0ca6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:36.447483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:36.472270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:36.472270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.472270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.472270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.472750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:36.472750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:36.472750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:36.501428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 400/790] 10.222.0.50 () {56 vars in 1135 bytes} [Tue Aug 30 01:43:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:36.512270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:36.629796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72362f-9984-4523-8ad9-6007a6ec4995 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:36.645665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:36.666832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705b0aa-3926-459f-a487-94c4a7cccaf0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:36.679379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:36.679992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.680281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.680558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.702714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:36.703651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:36.704176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:36.704755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.705306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.705781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.708790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:36.717952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:36.723061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:36.724001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:36.724967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:36.724967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:36.724967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:36.726024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:36.726921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:36.727782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:36.730354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 391/791] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:36.732434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 401/792] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:36.738400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:36.744293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:36.874111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d1f5-0643-4f41-8941-147cc8bbaa65 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:36.893478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325f9be-294e-49bc-a890-c4efeb10952a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:36.922246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b518a5-7cab-4c78-b1db-77218db78e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:36.923120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b518a5-7cab-4c78-b1db-77218db78e79 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.923395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b518a5-7cab-4c78-b1db-77218db78e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.923816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b518a5-7cab-4c78-b1db-77218db78e79 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.924800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b518a5-7cab-4c78-b1db-77218db78e79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:36.929941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b518a5-7cab-4c78-b1db-77218db78e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:36.931374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b518a5-7cab-4c78-b1db-77218db78e79 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:36.932775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee068-79d5-4848-abef-2be358e7a476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:36.933382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee068-79d5-4848-abef-2be358e7a476 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.933661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee068-79d5-4848-abef-2be358e7a476 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.933931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee068-79d5-4848-abef-2be358e7a476 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.934571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ee068-79d5-4848-abef-2be358e7a476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:36.934872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ee068-79d5-4848-abef-2be358e7a476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:36.936208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ee068-79d5-4848-abef-2be358e7a476 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:36.945382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 402/793] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:36.946752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2e43-84f7-487a-a0ca-1dffbc25619d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.947035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2e43-84f7-487a-a0ca-1dffbc25619d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.947305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2e43-84f7-487a-a0ca-1dffbc25619d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.948817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 403/794] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:36.949514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2e43-84f7-487a-a0ca-1dffbc25619d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:36.953265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 392/795] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:36.955053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267423-e5ef-431f-9772-68df3047db6e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:36.955339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267423-e5ef-431f-9772-68df3047db6e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:36.955615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267423-e5ef-431f-9772-68df3047db6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:36.974043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-79267423-e5ef-431f-9772-68df3047db6e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:36.975760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-79267423-e5ef-431f-9772-68df3047db6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:36.983208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:37.029034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:37.052381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 393/796] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:37.053859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267423-e5ef-431f-9772-68df3047db6e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.069296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2e43-84f7-487a-a0ca-1dffbc25619d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.109905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.111867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.113118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.113724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.127320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:37.129127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:37.145754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:37.146707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:37.147586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:37.147586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:37.147586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:37.149894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 404/797] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:37.155698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.234225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267423-e5ef-431f-9772-68df3047db6e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.373992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:37.386542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeeba6-8e40-43a6-949e-8434b694a257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.387167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeeba6-8e40-43a6-949e-8434b694a257 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.387443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeeba6-8e40-43a6-949e-8434b694a257 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.387713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeeba6-8e40-43a6-949e-8434b694a257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.394867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeeba6-8e40-43a6-949e-8434b694a257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:37.396165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeeba6-8e40-43a6-949e-8434b694a257 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:37.398688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d460b1-5ba8-43bb-a0b0-91e8220e4e28 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-eeaeeba6-8e40-43a6-949e-8434b694a257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:37.408189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:37.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:37.410064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 394/798] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:37.411037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fd9fa-0c8f-4366-9671-61e83d5bc8d0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.411206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fd9fa-0c8f-4366-9671-61e83d5bc8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.411466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fd9fa-0c8f-4366-9671-61e83d5bc8d0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.413036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 395/799] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:37.413685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fd9fa-0c8f-4366-9671-61e83d5bc8d0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.439103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.439925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.440208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.440555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.441199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:37.441500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:37.443105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:37.456214 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 405/800] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:37.457114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.467419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:37.524910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fd9fa-0c8f-4366-9671-61e83d5bc8d0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.558454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2613bce-6258-44c7-a215-33bd43bfc542 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.561783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763db2b-71e0-4333-b791-e359418d1c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.562814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763db2b-71e0-4333-b791-e359418d1c21 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.563280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763db2b-71e0-4333-b791-e359418d1c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.563736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763db2b-71e0-4333-b791-e359418d1c21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.575048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763db2b-71e0-4333-b791-e359418d1c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9763db2b-71e0-4333-b791-e359418d1c21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:37.579757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:37.580836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:37.581677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 396/801] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:37.582454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f63c46-191c-4992-8b7d-2e4796b1bcf5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.582689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f63c46-191c-4992-8b7d-2e4796b1bcf5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.583048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f63c46-191c-4992-8b7d-2e4796b1bcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.595502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.596095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.596402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.596667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.597352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:37.597648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:37.599055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:37.600378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-96f63c46-191c-4992-8b7d-2e4796b1bcf5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:37.601545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-96f63c46-191c-4992-8b7d-2e4796b1bcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:37.608194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:37.612324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 406/802] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:37.613774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.666601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 397/803] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:37.667741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dc0dc-7e19-4c5d-b7f9-98440a4cd36c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.668017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dc0dc-7e19-4c5d-b7f9-98440a4cd36c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.668313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dc0dc-7e19-4c5d-b7f9-98440a4cd36c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.669515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 398/804] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:37.670202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dc0dc-7e19-4c5d-b7f9-98440a4cd36c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.708069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:37.733003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48968-b60f-4d68-84be-8de97750a853 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.766979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.767707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.767986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.768272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.775087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:37.776184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dc0dc-7e19-4c5d-b7f9-98440a4cd36c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.776538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:37.789050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:37.789991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 407/805] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:37.790909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.817370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749dc3c-1e5d-4479-bc83-9f337fc4f4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.818083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749dc3c-1e5d-4479-bc83-9f337fc4f4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.818334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749dc3c-1e5d-4479-bc83-9f337fc4f4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.818803 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749dc3c-1e5d-4479-bc83-9f337fc4f4e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.819677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749dc3c-1e5d-4479-bc83-9f337fc4f4e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:37.820033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749dc3c-1e5d-4479-bc83-9f337fc4f4e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:37.821949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749dc3c-1e5d-4479-bc83-9f337fc4f4e3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:37.830708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 399/806] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:37.831865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e924-3abf-4dd7-ae4b-85b987b582c9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.832146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e924-3abf-4dd7-ae4b-85b987b582c9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.832465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e924-3abf-4dd7-ae4b-85b987b582c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.851055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2a51e924-3abf-4dd7-ae4b-85b987b582c9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:37.852392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2a51e924-3abf-4dd7-ae4b-85b987b582c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:37.854320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:37.894665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da66ce8-74fa-492f-9f10-378e8aec05c5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:37.931219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 400/807] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:37.932074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51e924-3abf-4dd7-ae4b-85b987b582c9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.942388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:37.943049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:37.943325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:37.943595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/5956fdd8d6b54640970cde2705a83260/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:37.945796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=5956fdd8d6b54640970cde2705a83260, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:37.947224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:37.967320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 408/808] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:37 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/5956fdd8d6b54640970cde2705a83260/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:37.968467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:37.993522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.083197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048e86c-61ca-45a8-b49c-c4563fa98563 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.085254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51e924-3abf-4dd7-ae4b-85b987b582c9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.106480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.125179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.125817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.126090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.126358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.126748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.127643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.128129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.128627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.129545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:38.130040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.131705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.137566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.139059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:38.154196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:38.155164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:38.156108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:38.156108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:38.156108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:38.156108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 409/809] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:38.156812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.160658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 401/810] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:43:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:38.162387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.261584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ce76d-ba75-4cc5-a876-b3280cd5a804 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.263344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.282917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8e2a7-317d-4bc9-ada5-6bd354070eb2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.305121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.305763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.306039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.306310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.306892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.308707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:38.317969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:38.319455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:38.325389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 410/811] 10.222.0.50 () {56 vars in 1129 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:38.326168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.326248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.326248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.326248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.326248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.326513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:38.326513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.326643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.339053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 402/812] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:38.339719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.367965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.442583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137c472-166a-4f64-b682-208edcd44ee2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.454331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd09f5-0378-4714-b604-5d9f66e34ac3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.477579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.478268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.479053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.479401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.495421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.496047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.496356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.496634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] PATH_INFO: `/v3/services/network` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.497204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.498553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:38.509505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:38.510528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 01:43:38.511416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:38.515476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 403/813] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:38.516232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.532353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.554659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.556106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.589937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.627990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04898c39-c8e5-4261-9dfd-2468d381feae None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.665778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 411/814] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:43:38.666817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.671715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.672419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.672761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.673097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.673790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:38.674158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.675575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.681080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 404/815] 10.222.0.50 () {56 vars in 1124 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:38.681786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.754312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.781589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd505da1-66f4-4fa0-b95b-956a17796563 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.790589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46da97-4315-4a71-83bb-ba37fadb7a20 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.816555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.817189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.817432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.817703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.818326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] RBAC: Adding query filter params (name=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:38.819182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.820566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.825905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 405/816] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:38.826775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.843720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.844807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.845224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:38.845850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:38.849114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:38.856819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:38.858938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:38.882505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:38.883661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:38.884837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:38.885466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 412/817] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:38.886647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:38.949378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8414-c326-48b5-ace8-caec8acea51e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:38.997834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:38.998940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:38.999386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.000069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.009064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17854bad-5dd4-492b-8757-59df0671cdb8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.009720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:39.016130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.018291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:39.038697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:39.039893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:39.041245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:39.041245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:39.041245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:39.041245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 406/818] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:39.042265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.055110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.056735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.057726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.058269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.059455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:39.060540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.062379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.072700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 413/819] 10.222.0.50 () {56 vars in 1124 bytes} [Tue Aug 30 01:43:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:39.078379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.145340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:39.190474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37093313-e8d9-4ce3-a5cf-3ddae17c8129 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.191600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee512f-b8b7-448f-827c-0637eef7a2f5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.225025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.225863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.226147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.226420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.227134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:39.227469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.228934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.231341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.231936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.232212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.232513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.233139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] RBAC: Adding query filter params (type=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:39.233436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.234767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.245298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 407/820] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:39.245298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.250304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 414/821] 10.222.0.50 () {56 vars in 1149 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:39.254754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.310368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:39.356958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767f68b-f93c-4786-bb27-8ae798d756df None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.358230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6ed21-b3f4-4733-b6f9-3381b636f0ab None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.377375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:39.394436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.395323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.395865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.396177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.396557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.397411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.397684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.397952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.398486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:39.398591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.399047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.399950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.400807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.419950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 408/822] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:39.421078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.421528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 415/823] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:43:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:39.422345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.545942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ad33b-06d8-4bb3-bda9-df1d20b5d2bb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.564865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:39.592042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd28cc-fac8-4dae-be2e-cfdc56a1e71e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.595799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.596558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.596989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.597291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.598087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.600524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.627367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 409/824] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:43:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:39.628205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.649904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.651354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.651888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.652367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.667456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.669876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:39.689241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:39.690365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:39.691227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:39.691227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:39.691227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:39.693296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 416/825] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:39.700286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.777148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:39.778183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a4f4-b1ac-4bdb-ace0-d1a0322f3e27 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.824674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.825731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.826241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.826908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.837558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.840031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:39.858209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:39.859133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:39.860013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:39.863198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 410/826] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:39.868660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.898809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fba2e8-3853-4502-bb6a-6f14ca7cff34 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:39.902525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:39.940142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0d1c4-bf94-4db3-9089-afb113052aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:39.941031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0d1c4-bf94-4db3-9089-afb113052aec None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.941483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0d1c4-bf94-4db3-9089-afb113052aec None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.941897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0d1c4-bf94-4db3-9089-afb113052aec None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.942841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0d1c4-bf94-4db3-9089-afb113052aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:39.943286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0d1c4-bf94-4db3-9089-afb113052aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:39.945389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0d1c4-bf94-4db3-9089-afb113052aec None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:39.958280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 417/827] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:39.963721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140cb13-c756-48f1-bcf1-362cf27daa23 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:39.963721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140cb13-c756-48f1-bcf1-362cf27daa23 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:39.963721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140cb13-c756-48f1-bcf1-362cf27daa23 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:39.963721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 418/828] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:39.967062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140cb13-c756-48f1-bcf1-362cf27daa23 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:39.997117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785b27-bae0-4ad0-839b-05febb7e3693 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.035583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f865550-1b07-4b01-aeed-e82064099d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.036197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f865550-1b07-4b01-aeed-e82064099d1d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.036645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f865550-1b07-4b01-aeed-e82064099d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.036923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f865550-1b07-4b01-aeed-e82064099d1d None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.037762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f865550-1b07-4b01-aeed-e82064099d1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:40.038069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f865550-1b07-4b01-aeed-e82064099d1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:40.039486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f865550-1b07-4b01-aeed-e82064099d1d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:40.056296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 411/829] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:40.056505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:40.056593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9681e-f0d0-4bfb-973a-14b8fbe67093 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.056593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9681e-f0d0-4bfb-973a-14b8fbe67093 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.056593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9681e-f0d0-4bfb-973a-14b8fbe67093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.073646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-84f9681e-f0d0-4bfb-973a-14b8fbe67093 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:40.074898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-84f9681e-f0d0-4bfb-973a-14b8fbe67093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:40.080897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140cb13-c756-48f1-bcf1-362cf27daa23 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.116416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.117049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.117328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.117603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.124666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:40.126125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:40.138077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:40.138953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:40.139831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 419/830] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:40.140761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:40.151367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 412/831] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:40.152295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9681e-f0d0-4bfb-973a-14b8fbe67093 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:40.169452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:40.261975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9681e-f0d0-4bfb-973a-14b8fbe67093 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.278105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131babc4-dc95-49ee-b341-78d4cb36526e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.308580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.309280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.309626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.309970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.310726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:40.311233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:40.312559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:40.323748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.324590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.325108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.325576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.326499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=95b365b4794c4edebe306e09336c42f3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:40.327008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:40.328726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:40.330736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 413/832] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:40.331507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:40.342412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 420/833] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:40 2022] GET /identity/v3/role_assignments?user.id=95b365b4794c4edebe306e09336c42f3&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:40.419679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:40.431881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:40.448020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734a30b-c76c-4382-9030-0710acdfa313 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.453037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:40.486369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.487286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.487556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.487829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/b5ef7dccfc5e43be993e201a19730a1a/roles/3aa65bdcfd1244cb9d358fa12c4c02cc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.507976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=b5ef7dccfc5e43be993e201a19730a1a, role_id=3aa65bdcfd1244cb9d358fa12c4c02cc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:40.509918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:40.524969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 414/834] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:40 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/b5ef7dccfc5e43be993e201a19730a1a/roles/3aa65bdcfd1244cb9d358fa12c4c02cc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:40.558606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1729f-f127-4134-a341-25427765f3d5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.598201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.598845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.599121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.599365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.612435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:40.618702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:40.619661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:40.620526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 421/835] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:40.626356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:40.743498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24e8ff-828f-4b03-9e0a-aa29eb4fe3f5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.773579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:40.779979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.780629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.780908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.781190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.781875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:40.782390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:40.783934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:40.793622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 415/836] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:40.799346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:40.864140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:40.905160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f396d-ceee-42ad-8745-e84a01cbb8dc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:40.939896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:40.940918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:40.941445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:40.941916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:40.954052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:40.955794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:40.971000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:40.971956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:40.972889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:40.972889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:40.972889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:40.975456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 422/837] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:40.981629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:41.089418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51e1ea-1331-42cc-9fb8-f54d41967037 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:41.126440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:41.127050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:41.127334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:41.127613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:41.128341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:41.128651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:41.130115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:41.143497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 416/838] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:41.149083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:41.255439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bcf8f-d390-4825-8959-9d9112f4bf5c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:41.270425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:41.299284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:41.300160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:41.300720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:41.301217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:41.309304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:41.311004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:41.322899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:41.323817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 01:43:41.324889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:41.327299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 423/839] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:43:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:41.333384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:41.386807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:41.441596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de267d-bddf-4c67-9c47-37895ff489d7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:41.478565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158185-3a52-4609-bfc2-362adb7f1f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:41.479537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158185-3a52-4609-bfc2-362adb7f1f44 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:41.480050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158185-3a52-4609-bfc2-362adb7f1f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:41.480557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158185-3a52-4609-bfc2-362adb7f1f44 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:41.481484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158185-3a52-4609-bfc2-362adb7f1f44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:41.482015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158185-3a52-4609-bfc2-362adb7f1f44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:41.483625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158185-3a52-4609-bfc2-362adb7f1f44 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:41.502536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 417/840] 10.222.0.50 () {56 vars in 1135 bytes} [Tue Aug 30 01:43:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:41.509045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:41.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5e4fe-c184-41a4-9487-a580a4654b12 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:41.509974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5e4fe-c184-41a4-9487-a580a4654b12 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:41.510252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5e4fe-c184-41a4-9487-a580a4654b12 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:41.511794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 418/841] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:41.516869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d57c10-d9c3-49cd-8ced-62476a4c60c6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:41.517153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d57c10-d9c3-49cd-8ced-62476a4c60c6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:41.517434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d57c10-d9c3-49cd-8ced-62476a4c60c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:41.538067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-58d57c10-d9c3-49cd-8ced-62476a4c60c6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:41.539766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-58d57c10-d9c3-49cd-8ced-62476a4c60c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:41.632110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 419/842] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:41.645813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721aa9d-2ce2-4dd2-b2f7-f095790f69da None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:41.682995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:41.683979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:41.684493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:41.684968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:41.730153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:41.756340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:41.758063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:41.859477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 424/843] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:43:41.866288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d57c10-d9c3-49cd-8ced-62476a4c60c6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:41.891884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:41.906465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:41.968624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:41.993179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d57c10-d9c3-49cd-8ced-62476a4c60c6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.003897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3c8c7-4db7-4f5b-b6ef-4097d8f6d7c4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.038126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.039286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.039743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.040463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.048986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.049746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.050117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.050675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.051153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.052736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.055530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.057456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:42.062043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:42.063250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:42.064310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:42.071002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 425/844] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:42.072300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:42.078901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:42.079843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:42.080778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:42.080778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:42.080778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:42.087825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 420/845] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:42.088944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.186517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b51164-bfdb-4022-821a-328b6de32a7a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.217181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:42.222999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.223758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.224042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.224364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.225539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:42.225826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.227298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.234982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aba6b9-76f9-4227-8871-d4b8ee952510 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.236360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 426/846] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:42.238879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.242421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:42.282053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.283035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.291003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.298065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.299380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:42.299914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.302051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.324396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 421/847] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:42.326873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.359890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d207015-84a1-4760-8ee7-be4a381cf464 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.410947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ac247-6eee-4bdd-b72e-669a29f39276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.412793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ac247-6eee-4bdd-b72e-669a29f39276 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.414223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ac247-6eee-4bdd-b72e-669a29f39276 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.414722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ac247-6eee-4bdd-b72e-669a29f39276 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.415649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ac247-6eee-4bdd-b72e-669a29f39276 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:42.416157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ac247-6eee-4bdd-b72e-669a29f39276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.417771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ac247-6eee-4bdd-b72e-669a29f39276 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.424160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 427/848] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:42.425254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eee27c-1137-44ce-8ed4-212752b7e0b6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.425541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eee27c-1137-44ce-8ed4-212752b7e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.425817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eee27c-1137-44ce-8ed4-212752b7e0b6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.428504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 428/849] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:42.433151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad46fa-2a09-4b55-b72a-b84b604381b5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.433777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad46fa-2a09-4b55-b72a-b84b604381b5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.434256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad46fa-2a09-4b55-b72a-b84b604381b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.450392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556cacbb-e519-4c68-8fc5-12dbafdf5684 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.455044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5fad46fa-2a09-4b55-b72a-b84b604381b5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:42.456307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5fad46fa-2a09-4b55-b72a-b84b604381b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:42.498146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c271de5-e4c2-47cb-88fa-79d4230e3393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.498791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c271de5-e4c2-47cb-88fa-79d4230e3393 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.499066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c271de5-e4c2-47cb-88fa-79d4230e3393 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.499337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c271de5-e4c2-47cb-88fa-79d4230e3393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.506863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c271de5-e4c2-47cb-88fa-79d4230e3393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.508340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c271de5-e4c2-47cb-88fa-79d4230e3393 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7c271de5-e4c2-47cb-88fa-79d4230e3393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:42.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:42.524894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:42.525764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:42.526853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 422/850] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:42.527110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ffddb-a56f-4684-b92b-4efe399856b7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.527386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ffddb-a56f-4684-b92b-4efe399856b7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.527655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ffddb-a56f-4684-b92b-4efe399856b7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.533334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 423/851] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:42.533334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ffddb-a56f-4684-b92b-4efe399856b7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.535038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:42.540649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 429/852] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:42.542344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3b4a9-7b9c-474a-811f-327440968f13 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.543454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3b4a9-7b9c-474a-811f-327440968f13 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.543946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3b4a9-7b9c-474a-811f-327440968f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.561753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:42.569335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f1e3b4a9-7b9c-474a-811f-327440968f13 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:42.571249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f1e3b4a9-7b9c-474a-811f-327440968f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:42.661064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ffddb-a56f-4684-b92b-4efe399856b7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.675686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 430/853] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:42.705032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.705946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.706423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.706954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.707900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:42.708472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.710641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.723594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 424/854] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:42.736347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3b4a9-7b9c-474a-811f-327440968f13 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.839784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3b4a9-7b9c-474a-811f-327440968f13 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:42.845148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.868321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:42.878274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:42.879120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:42.879395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:42.879667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:42.880423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=5956fdd8d6b54640970cde2705a83260) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:42.881535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:42.883763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:42.891191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 431/855] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/role_assignments?user.id=5956fdd8d6b54640970cde2705a83260&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:42.928227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:42.945568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:42.967098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3211-fbf6-41e7-8cfc-9e87341e5644 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.003712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.004471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.004752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.005020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.005607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:43.007516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:43.017539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:43.018519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:43.019403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 425/856] 10.222.0.50 () {56 vars in 1132 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:43.026703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:43.061043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa46ef8-c8e9-424a-80cc-f1249b9b08de None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.103628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.104925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.105225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.105492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.116889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:43.122119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:43.123225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:43.124265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 432/857] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:43:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:43.129433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:43.147357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53ef84-271a-4e33-a0f9-2333aabbc67a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.180172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:43.186371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.187152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.187598 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.188010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.188919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:43.189357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:43.190936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:43.197201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 426/858] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:43.198749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:43.208063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:43.258689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee36f91-b45f-493a-bd2f-7660b3e59165 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.302354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.304136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.304546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.304937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.306179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:43.306584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:43.308877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:43.322332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 433/859] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:43.327522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:43.334581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646d651-e1d3-4026-b2b0-ee179088974e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.369980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.370579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.370856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.371124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.371848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:43.372150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:43.373532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:43.382993 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 427/860] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:43.454942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe41bc-11f9-43eb-8112-eb74bda44e4f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.470279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:43.494048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.494713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.494994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.495280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.507989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:43.509634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:43.525619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:43.526615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:43.527578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:43.527578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:43.527578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:43.527578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 434/861] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:43.533277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:43.598506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:43.643182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7419a-cad2-4c4b-9e44-ff253e382c6a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.677949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.679025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.679297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.679563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.680190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:43.680513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:43.681865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:43.694313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 428/862] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:43.700267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:43.804156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432356e-0f8d-4777-975c-8f38d3fef3de None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.838478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:43.839162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:43.839438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:43.839708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:43.847579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:43.849044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:43.861918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:43.862855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:43.863763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 435/863] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:43:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:43.871141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:43.974134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb9ca-92a2-4078-9830-af134a9df7f5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:43.996254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:44.011897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:44.012877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.013380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.013850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.014776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:44.015287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:44.016997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:44.036744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 429/864] 10.222.0.50 () {56 vars in 1149 bytes} [Tue Aug 30 01:43:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:44.043266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:44.070895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:44.148584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c513363-bdb1-4d8e-beff-9b32c32e3824 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:44.182064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cdcd1-62eb-40cd-975d-c3f5cf18d25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:44.183038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cdcd1-62eb-40cd-975d-c3f5cf18d25a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.183538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cdcd1-62eb-40cd-975d-c3f5cf18d25a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.184004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cdcd1-62eb-40cd-975d-c3f5cf18d25a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.249952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cdcd1-62eb-40cd-975d-c3f5cf18d25a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:44.251709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cdcd1-62eb-40cd-975d-c3f5cf18d25a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:44.279165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:44.357263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 436/865] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:43:44.363018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:44.463836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2814b8b-866d-4224-81b8-29b627fee0e3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:44.499931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:44.500635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.500909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.501178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.512052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:44.513531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:44.520470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef347-b00d-41f7-b961-011987dcf604 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.521059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef347-b00d-41f7-b961-011987dcf604 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.522241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef347-b00d-41f7-b961-011987dcf604 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.524057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 437/866] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:44.529235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c24b7b-b20b-4601-bcce-c2f440ee8ac7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.529765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c24b7b-b20b-4601-bcce-c2f440ee8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:44.529899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:44.530748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:44.531638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:44.531638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:44.531638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:44.531638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 430/867] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:44.532727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c24b7b-b20b-4601-bcce-c2f440ee8ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.536721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:44.552474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a6c24b7b-b20b-4601-bcce-c2f440ee8ac7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:44.554321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a6c24b7b-b20b-4601-bcce-c2f440ee8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:44.570274 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:44.652374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 438/868] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:44.654926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c0d2-a836-4149-901b-7e5d19e416e7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:44.690978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:44.691896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.692344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.692756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.694192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:44.694636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:44.696133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:44.712322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 431/869] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:44.713978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c24b7b-b20b-4601-bcce-c2f440ee8ac7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:44.775266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:44.820739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c24b7b-b20b-4601-bcce-c2f440ee8ac7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:44.855012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc83054-ebab-44b7-a1e4-a61dac120b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:44.855640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83054-ebab-44b7-a1e4-a61dac120b6f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.855915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83054-ebab-44b7-a1e4-a61dac120b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.856183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc83054-ebab-44b7-a1e4-a61dac120b6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.863332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc83054-ebab-44b7-a1e4-a61dac120b6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:44.864805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc83054-ebab-44b7-a1e4-a61dac120b6f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2cc83054-ebab-44b7-a1e4-a61dac120b6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:44.876475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:44.877350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:44.878220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 439/870] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:44.885139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:44.899851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727ae5c-c422-41e0-ada4-6e740a009cc7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.900142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727ae5c-c422-41e0-ada4-6e740a009cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.900430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727ae5c-c422-41e0-ada4-6e740a009cc7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.901988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 440/871] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:44.906799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e323d6-f12b-42f6-b343-ced7f9ef57fb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:44.907030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e323d6-f12b-42f6-b343-ced7f9ef57fb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:44.907302 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e323d6-f12b-42f6-b343-ced7f9ef57fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:44.928969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f4e323d6-f12b-42f6-b343-ced7f9ef57fb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:44.930141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f4e323d6-f12b-42f6-b343-ced7f9ef57fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:44.993726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:45.004046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bebc5d-cbaf-4198-a450-2fb22cad5703 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:45.008278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 441/872] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:45.008278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e323d6-f12b-42f6-b343-ced7f9ef57fb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:45.058585 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:45.059500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.059920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.061024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.061905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:45.062347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:45.064793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:45.067395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:45.087497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 432/873] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:45.088267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:45.144262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e323d6-f12b-42f6-b343-ced7f9ef57fb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:45.187989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef237584-6840-44af-9801-385fada8054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:45.188678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef237584-6840-44af-9801-385fada8054b None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.188961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef237584-6840-44af-9801-385fada8054b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.189232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef237584-6840-44af-9801-385fada8054b None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.189881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef237584-6840-44af-9801-385fada8054b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:45.191242 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef237584-6840-44af-9801-385fada8054b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:45.216277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 442/874] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:43:45 2022] POST /identity/v3/services => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:45.242873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372bb16-2702-4c21-992b-951d8651b5aa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:45.288402 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd38896-be94-403e-97e4-9f6b084f633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:45.289019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd38896-be94-403e-97e4-9f6b084f633a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.289295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd38896-be94-403e-97e4-9f6b084f633a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.289564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd38896-be94-403e-97e4-9f6b084f633a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.290356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd38896-be94-403e-97e4-9f6b084f633a None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=b5ef7dccfc5e43be993e201a19730a1a) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:45.290664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd38896-be94-403e-97e4-9f6b084f633a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:45.292176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd38896-be94-403e-97e4-9f6b084f633a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:45.299436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 433/875] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:45 2022] GET /identity/v3/role_assignments?user.id=b5ef7dccfc5e43be993e201a19730a1a&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:45.327671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef237584-6840-44af-9801-385fada8054b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:45.451269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef237584-6840-44af-9801-385fada8054b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:45.453744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:45.479112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f0723-a38f-4ab7-9819-0ec4778c394b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.479401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f0723-a38f-4ab7-9819-0ec4778c394b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.479674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f0723-a38f-4ab7-9819-0ec4778c394b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.482094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 434/876] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:45.487403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09db007-5bb2-4e58-a727-7b140ce75f50 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.487843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09db007-5bb2-4e58-a727-7b140ce75f50 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.488120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09db007-5bb2-4e58-a727-7b140ce75f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.493896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:45.494543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.494824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.495102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.495692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:45.497112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:45.507143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:45.508135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:45.509681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 443/877] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:45.510679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f09db007-5bb2-4e58-a727-7b140ce75f50 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:45.511549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f09db007-5bb2-4e58-a727-7b140ce75f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:45.514069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:45.515589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:45.600621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 435/878] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:45.634542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5265d8d-f97b-4193-8e0a-b513a4e76015 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:45.670101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:45.670709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.671044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.671322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.671944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:45.672264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:45.673624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:45.678600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 444/879] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:45.684175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09db007-5bb2-4e58-a727-7b140ce75f50 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:45.786824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09db007-5bb2-4e58-a727-7b140ce75f50 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:45.822714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ca8b-4786-4883-8d77-e7ba5252cd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:45.823379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ca8b-4786-4883-8d77-e7ba5252cd3d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:45.823694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ca8b-4786-4883-8d77-e7ba5252cd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:45.823968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ca8b-4786-4883-8d77-e7ba5252cd3d None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:45.824679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ca8b-4786-4883-8d77-e7ba5252cd3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:45.824978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ca8b-4786-4883-8d77-e7ba5252cd3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:45.826480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ca8b-4786-4883-8d77-e7ba5252cd3d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:45.831490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 436/880] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:45.885141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:45.920900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:46.001203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a1ffd-c16a-4cdb-8fa6-477690c71a9c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:46.038427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf0742-353c-43e3-92ca-edb0b291e92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:46.038949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf0742-353c-43e3-92ca-edb0b291e92f None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:46.039221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf0742-353c-43e3-92ca-edb0b291e92f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:46.039493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf0742-353c-43e3-92ca-edb0b291e92f None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:46.040194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf0742-353c-43e3-92ca-edb0b291e92f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:46.041593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf0742-353c-43e3-92ca-edb0b291e92f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:46.060282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 445/881] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:43:45 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:47.227158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d5e38-3b69-49fd-ab13-1c85ba6c37a8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.227680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d5e38-3b69-49fd-ab13-1c85ba6c37a8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.227791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d5e38-3b69-49fd-ab13-1c85ba6c37a8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:47.229620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 437/882] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:47.234785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32fe16-a8a5-490a-a905-a8c2761de142 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.235177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32fe16-a8a5-490a-a905-a8c2761de142 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.235335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32fe16-a8a5-490a-a905-a8c2761de142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:47.254302 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bf32fe16-a8a5-490a-a905-a8c2761de142 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:47.255474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bf32fe16-a8a5-490a-a905-a8c2761de142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:47.330357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 446/883] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:47.437873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9dad3-ed6d-49eb-bda8-b704acf8a982 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.438576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9dad3-ed6d-49eb-bda8-b704acf8a982 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.439056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9dad3-ed6d-49eb-bda8-b704acf8a982 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:47.440890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 438/884] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:47.446667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b76f-8c36-49ae-bb3f-2058afd9aa49 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.447231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b76f-8c36-49ae-bb3f-2058afd9aa49 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.447715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b76f-8c36-49ae-bb3f-2058afd9aa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:47.466931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0196b76f-8c36-49ae-bb3f-2058afd9aa49 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:47.468463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0196b76f-8c36-49ae-bb3f-2058afd9aa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:47.482347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:47.538774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 447/885] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:47.599252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9dad3-ed6d-49eb-bda8-b704acf8a982 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:47.603327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:47.707989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9dad3-ed6d-49eb-bda8-b704acf8a982 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:47.744060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:47.744634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.744911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.745182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:47.745764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:47.747111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:47.756732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:47.757725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:47.758537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:47.762762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 439/886] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:47.764116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196b76f-8c36-49ae-bb3f-2058afd9aa49 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:47.823155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:47.873735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196b76f-8c36-49ae-bb3f-2058afd9aa49 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:47.904817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:47.909763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdce801-00ce-4112-89cd-947e8aacffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:47.910379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdce801-00ce-4112-89cd-947e8aacffbd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.910657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdce801-00ce-4112-89cd-947e8aacffbd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.910935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdce801-00ce-4112-89cd-947e8aacffbd None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:47.911570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdce801-00ce-4112-89cd-947e8aacffbd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:47.911874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdce801-00ce-4112-89cd-947e8aacffbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:47.913330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdce801-00ce-4112-89cd-947e8aacffbd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:47.924461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 448/887] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:47.924461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf7998-0e6c-4bbb-be8b-770ce6b221c4 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.924461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf7998-0e6c-4bbb-be8b-770ce6b221c4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.924461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf7998-0e6c-4bbb-be8b-770ce6b221c4 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:47.924461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 449/888] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:47.924461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf7998-0e6c-4bbb-be8b-770ce6b221c4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:47.941395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:47.945412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a918d8c-c29c-45f4-91e5-12764424c81e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:47.991824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3c0ec-26fc-4cc5-8840-82dd7451505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:47.992502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3c0ec-26fc-4cc5-8840-82dd7451505e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:47.992757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3c0ec-26fc-4cc5-8840-82dd7451505e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:47.993051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3c0ec-26fc-4cc5-8840-82dd7451505e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.005408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3c0ec-26fc-4cc5-8840-82dd7451505e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0ac3c0ec-26fc-4cc5-8840-82dd7451505e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:48.011270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:48.012438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:43:48.013403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:48.016523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 440/889] 10.222.0.50 () {56 vars in 1120 bytes} [Tue Aug 30 01:43:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:48.018230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33fd0-eecc-40a3-b541-9a4497a2aa5a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.018517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33fd0-eecc-40a3-b541-9a4497a2aa5a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.019001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33fd0-eecc-40a3-b541-9a4497a2aa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.036216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf7998-0e6c-4bbb-be8b-770ce6b221c4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.036770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-83c33fd0-eecc-40a3-b541-9a4497a2aa5a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:48.038438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-83c33fd0-eecc-40a3-b541-9a4497a2aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:48.071541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9912c6-72e3-412e-94d7-f841244255a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.072341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9912c6-72e3-412e-94d7-f841244255a7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.073340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9912c6-72e3-412e-94d7-f841244255a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.073634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9912c6-72e3-412e-94d7-f841244255a7 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.074265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9912c6-72e3-412e-94d7-f841244255a7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:48.074573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9912c6-72e3-412e-94d7-f841244255a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.075955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9912c6-72e3-412e-94d7-f841244255a7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.085662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 450/890] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:48.085662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808470-a9af-4f03-bef9-1ab525fe454e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.085662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808470-a9af-4f03-bef9-1ab525fe454e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.085662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808470-a9af-4f03-bef9-1ab525fe454e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.085662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 451/891] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:48.085662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808470-a9af-4f03-bef9-1ab525fe454e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.109380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 441/892] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:48.110209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33fd0-eecc-40a3-b541-9a4497a2aa5a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.164167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.182907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.188985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808470-a9af-4f03-bef9-1ab525fe454e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.222836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0f662-6d18-44ec-a20b-076db3c605f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.223431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0f662-6d18-44ec-a20b-076db3c605f4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.223708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0f662-6d18-44ec-a20b-076db3c605f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.223972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0f662-6d18-44ec-a20b-076db3c605f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.225171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0f662-6d18-44ec-a20b-076db3c605f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:48.225467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0f662-6d18-44ec-a20b-076db3c605f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.226808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0f662-6d18-44ec-a20b-076db3c605f4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.229081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33fd0-eecc-40a3-b541-9a4497a2aa5a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.235462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 452/893] 10.222.0.50 () {56 vars in 1115 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:48.236834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf40a79-b566-4bab-8ced-3faf823727ca None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.237115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf40a79-b566-4bab-8ced-3faf823727ca None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.237387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf40a79-b566-4bab-8ced-3faf823727ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.255932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2cf40a79-b566-4bab-8ced-3faf823727ca None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:48.257057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2cf40a79-b566-4bab-8ced-3faf823727ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:48.271199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.271836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.272115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.272421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.273118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] RBAC: Adding query filter params (interface=public, service_id=86eaf6d1b8254583b080f227292afa6f, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:48.273426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.274816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.280042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 442/894] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/endpoints?service_id=86eaf6d1b8254583b080f227292afa6f&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:48.280837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.336275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 453/895] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:48.336275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf40a79-b566-4bab-8ced-3faf823727ca None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.399927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583572e-f79b-440a-a723-2061f760d489 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.408848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.425201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.439039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.439697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.440025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.440318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.444860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf40a79-b566-4bab-8ced-3faf823727ca None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.451332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.452900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:48.469070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:48.469980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:48.470866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:48.470866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:48.470866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:48.470866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 443/896] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:48.471095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.484077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.484751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.485027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.485299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.485906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.487518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.500279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 454/897] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:43:48.500279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.601872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638535a-8175-4617-b6aa-850d5665777a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.629594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc6540-ba42-4467-9808-0151e33c0e5e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.642873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.643491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.643772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.644043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.644727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.646089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.664533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.667792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 444/898] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:43:48 2022] POST /identity/v3/services => generated 241 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:48.668722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.678739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.679395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.679667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.679934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.680739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:48.681046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.682580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.697402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 455/899] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:48.705378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.721217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.802105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f869ed-ba62-4b31-86cc-53fc62dde185 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.828486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13fcea-3c56-470a-a4d7-04fbe65498a2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:48.843969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.844797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.845111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.845377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.846073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.847423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:48.857144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:48.858150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:48.859012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 445/900] 10.222.0.50 () {56 vars in 1132 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:48.864171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.874872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:48.875633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:48.875987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:48.876384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:48.886132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:48.887674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:48.901828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:48.902803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:48.903721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 456/901] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:48.908499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:48.963804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.989219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:48.999140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2200-2a28-434e-9c88-96f8f463cefb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.028060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feb0c-959b-438e-bcd7-ab1a303948e6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.034493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.035106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.035389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.035663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.036320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:49.036627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.038885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.045082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 446/902] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:49.054635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:49.072983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.073590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.073863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.074132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.074861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:49.075163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.076632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.099099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 457/903] 10.222.0.50 () {56 vars in 1149 bytes} [Tue Aug 30 01:43:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:49.101341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:49.170801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464ce8-fcf5-4c58-a20c-00460e2c066a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.207334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.207995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.208297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.208571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.209749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:49.210041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.211437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.216648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 447/904] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:49.222404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:49.233616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b3a85-4a1c-4904-aca1-2bc26c6427d6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.243749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:49.268747 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:49.280773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.281837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.282688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.283173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.345624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeeac7-1637-435a-b13a-44b629fa1e24 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.363447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.364991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.394950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.396267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.397511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.398008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.398914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] RBAC: Adding query filter params (interface=public, service_id=f1d0ad14c1774b38b175566cc45f51fa, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:49.399584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.402058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.409165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 448/905] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:43:49 2022] GET /identity/v3/endpoints?service_id=f1d0ad14c1774b38b175566cc45f51fa&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:49.415622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:49.487490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 458/906] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:43:49.488916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:49.497224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:49.516941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:49.528938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b99883-4553-4d61-8d66-301589385c6e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.566435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.567351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.567801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.568842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.569614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.571249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.584838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 449/907] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:43:49 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:43:49.599787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeb980-55b3-4dcd-990e-882e53e7b08d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.641052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.641696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.641974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.642252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.654418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.655914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:49.672552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:49.673550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:49.674422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:49.674422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:49.674422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:49.674422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 459/908] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:49.679775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:49.792470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd178671-200c-465b-9e64-981993a1b67a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.830180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.831156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.831654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.832160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:49.833066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:49.833571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:49.835752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:49.850143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 450/909] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:49.856691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:49.916589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:49.958813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc8a89-558d-474e-a6bd-85933f242254 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:49.993734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:49.994378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:49.994665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:49.994947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.001769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:50.003242 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:50.016171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:50.017180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:50.018063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 460/910] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:50.025328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:50.056679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:50.136182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99929344-3a07-4b2e-8af0-bddcaebd4728 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:50.171143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d790f1b-82e1-4f26-9027-845166641d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:50.172092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d790f1b-82e1-4f26-9027-845166641d5d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:50.172610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d790f1b-82e1-4f26-9027-845166641d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:50.173144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d790f1b-82e1-4f26-9027-845166641d5d None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.174040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d790f1b-82e1-4f26-9027-845166641d5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:50.174547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d790f1b-82e1-4f26-9027-845166641d5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:50.176192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d790f1b-82e1-4f26-9027-845166641d5d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:50.189760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 451/911] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:50.196717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:50.314616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c11dd7-77a1-45a6-9b45-e449af086bb3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:50.337098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:50.351970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508533e-c2ff-45f8-92d1-b220822da3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:50.352621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508533e-c2ff-45f8-92d1-b220822da3cb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:50.352923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508533e-c2ff-45f8-92d1-b220822da3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:50.353225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508533e-c2ff-45f8-92d1-b220822da3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.353980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508533e-c2ff-45f8-92d1-b220822da3cb None admin] RBAC: Adding query filter params (role.id=3c74c79658db45408d0892a3de5fcb85, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=b5ef7dccfc5e43be993e201a19730a1a) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:50.354291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508533e-c2ff-45f8-92d1-b220822da3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:50.355758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508533e-c2ff-45f8-92d1-b220822da3cb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:50.362127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 461/912] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:50 2022] GET /identity/v3/role_assignments?user.id=b5ef7dccfc5e43be993e201a19730a1a&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3c74c79658db45408d0892a3de5fcb85 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:50.454266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd489541-6a82-42c4-9b55-63beba43738f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:50.454796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd489541-6a82-42c4-9b55-63beba43738f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:50.455048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd489541-6a82-42c4-9b55-63beba43738f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.456822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 452/913] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:50.462774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775072-c122-40fe-adbb-1f85d7961172 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:50.463208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775072-c122-40fe-adbb-1f85d7961172 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:50.463588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77775072-c122-40fe-adbb-1f85d7961172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.484026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-77775072-c122-40fe-adbb-1f85d7961172 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:50.485419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-77775072-c122-40fe-adbb-1f85d7961172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:50.557385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 462/914] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:50.675368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c99e-5f80-414a-9aca-df2d7fd1d0d7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:50.675670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c99e-5f80-414a-9aca-df2d7fd1d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:50.675850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593c99e-5f80-414a-9aca-df2d7fd1d0d7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.677465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 453/915] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:50.683324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fd02a-3e28-422d-a95e-64f66cb5e1e5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:50.683837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fd02a-3e28-422d-a95e-64f66cb5e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:50.684343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fd02a-3e28-422d-a95e-64f66cb5e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.706782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cd9fd02a-3e28-422d-a95e-64f66cb5e1e5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:50.707954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cd9fd02a-3e28-422d-a95e-64f66cb5e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:50.784508 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 463/916] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:50.852469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593c99e-5f80-414a-9aca-df2d7fd1d0d7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:50.928957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:50.954337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593c99e-5f80-414a-9aca-df2d7fd1d0d7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:50.994632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:50.995266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:50.995547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:50.995820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:50.996459 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:50.997862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:51.007238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:51.008200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:51.009088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 454/917] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:51.014725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fd02a-3e28-422d-a95e-64f66cb5e1e5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.058789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:51.079392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.131901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fd02a-3e28-422d-a95e-64f66cb5e1e5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:51.174536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:51.175286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.175766 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.176223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.177180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:51.177675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:51.179893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.187607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 464/918] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:51.195402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.214103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae4bac-cea8-440d-8009-5ea57ebfbcfe None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:51.248540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:51.249169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.249460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.249719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.250293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:51.251650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:51.261259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:51.262284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:51.263149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 455/919] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:51.269106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.273954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:51.333984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b1b6a-9d70-4d2e-aa10-bfaa25da6aa4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:51.344829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:51.388103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:51.389614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.390427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.390948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.392482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:51.393116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:51.395080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.401660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d3cb7-4eb8-4583-b3df-de046a277ecb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:51.407529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 465/920] 10.222.0.50 () {56 vars in 1114 bytes} [Tue Aug 30 01:43:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:51.408841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.443095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:51.444222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.444528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.444936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.445617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:51.445921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:51.447611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.452831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 456/921] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:51.457955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.538395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:51.556608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b566f-3cbb-44a9-a8ff-a8d23c5dd0ad None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:51.587467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e98854-9889-447c-8a06-b4dae6b6ae16 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:51.592998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:51.593654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.593932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.594200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.594927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:51.596361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.624283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 466/922] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:43:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:43:51.645189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6627dd-7714-4e24-96aa-141a11e6073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:51.645922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6627dd-7714-4e24-96aa-141a11e6073c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.646265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6627dd-7714-4e24-96aa-141a11e6073c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.646642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6627dd-7714-4e24-96aa-141a11e6073c None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.647414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6627dd-7714-4e24-96aa-141a11e6073c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:51.647782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6627dd-7714-4e24-96aa-141a11e6073c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:51.649357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6627dd-7714-4e24-96aa-141a11e6073c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.655694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 457/923] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:51.661469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.667847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22ac20-d779-4ffe-b52b-538573e5ba97 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.668139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22ac20-d779-4ffe-b52b-538573e5ba97 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.668451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22ac20-d779-4ffe-b52b-538573e5ba97 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.669834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 458/924] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:51.675758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955536-5eff-4abc-ad81-5a113d57da0d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.676077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955536-5eff-4abc-ad81-5a113d57da0d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.676430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5955536-5eff-4abc-ad81-5a113d57da0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.701677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a5955536-5eff-4abc-ad81-5a113d57da0d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:51.702726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a5955536-5eff-4abc-ad81-5a113d57da0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:51.710246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:51.783466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 459/925] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:51.809415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56623e-8c10-470f-bf95-9fa6f9c2f254 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:51.847208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:51.847819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:51.848097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:51.848414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:51.849157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] RBAC: Adding query filter params (interface=public, service_id=fccf870e8f3649538dd02953fe175cd9, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:51.849992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:51.851365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:51.856898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 467/926] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:43:51 2022] GET /identity/v3/endpoints?service_id=fccf870e8f3649538dd02953fe175cd9&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:51.862372 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955536-5eff-4abc-ad81-5a113d57da0d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:51.924447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:51.973821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5955536-5eff-4abc-ad81-5a113d57da0d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:52.009384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:52.009996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.010275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.010549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.011156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:52.012551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:52.019165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 460/927] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:43:51 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:43:52.059322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:52.157771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:52.167997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39e06a-dee7-4000-b82e-0680e3a9ab10 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:52.206290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:52.206928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.207206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.207477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.208065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:52.210066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:52.219123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:52.220046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:52.220958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 468/928] 10.222.0.50 () {56 vars in 1132 bytes} [Tue Aug 30 01:43:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:52.228834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:52.342485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef884f42-56a9-4c3e-8c81-2b24bf1a4fed None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:52.366129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:52.381033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113dadd-b896-4b25-9e49-1541a656b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:52.381650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113dadd-b896-4b25-9e49-1541a656b84c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.381928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113dadd-b896-4b25-9e49-1541a656b84c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.382202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113dadd-b896-4b25-9e49-1541a656b84c None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.382837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113dadd-b896-4b25-9e49-1541a656b84c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:52.383135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113dadd-b896-4b25-9e49-1541a656b84c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:52.384526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113dadd-b896-4b25-9e49-1541a656b84c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:52.389807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 461/929] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:52.395347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:52.409029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba58b7-8587-4cbf-908e-640b2b7eab27 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.409316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba58b7-8587-4cbf-908e-640b2b7eab27 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.409589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba58b7-8587-4cbf-908e-640b2b7eab27 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.411198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 462/930] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:52.416043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c2dfa-36bc-4b4a-9064-6ed867049537 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.416366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c2dfa-36bc-4b4a-9064-6ed867049537 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.416647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c2dfa-36bc-4b4a-9064-6ed867049537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.443262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-631c2dfa-36bc-4b4a-9064-6ed867049537 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:52.444589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-631c2dfa-36bc-4b4a-9064-6ed867049537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:52.515061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16add10-5911-4d4e-a80b-0f083f6f3fb7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:52.516805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 463/931] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:52.551503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:52.552486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.553330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.553604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.554233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:52.554535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:52.555902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:52.561930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 469/932] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:52.567371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c2dfa-36bc-4b4a-9064-6ed867049537 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:52.666518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c2dfa-36bc-4b4a-9064-6ed867049537 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:52.681000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:52.703501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:52.704492 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.704980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.705476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.706344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:52.707891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:52.730425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 464/933] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:43:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:43:52.819801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:52.844157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:52.938356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d183a-b507-4ef9-89ab-592cf573858a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:52.980416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:52.981434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:52.981936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:52.982452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:52.994709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:52.999758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:53.000753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:43:53.001632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:53.004218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 470/934] 10.222.0.50 () {56 vars in 1120 bytes} [Tue Aug 30 01:43:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:53.012878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:53.118153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8ca74-0ca1-445d-87f1-770553e01505 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:53.153035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:53.154156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:53.154657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:53.155139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:53.155994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:53.156523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:53.158081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:53.166838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 465/935] 10.222.0.50 () {56 vars in 1115 bytes} [Tue Aug 30 01:43:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:53.172715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:53.254750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:53.282427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e355d6-17c1-4ccf-bdaa-bba4bd6e01cf None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:53.320794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:53.321499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:53.321780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:53.322053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:53.332457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:53.333973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:53.349532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:53.350482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:53.351357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:53.351357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:53.351357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:53.355429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 471/936] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:53.357329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:53.389957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:53.461838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11def5bd-02c9-45c9-90ce-5553adb45da4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:53.496015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:53.496998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:53.497509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:53.497983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:53.498829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:53.499333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:53.500906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:53.513790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 466/937] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:53.524361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:53.627855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fdce-fdad-4022-94da-c96f41da7f17 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:53.654343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:53.663496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9697fe-c2e7-452b-8bf0-88295f6ee140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:53.664186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9697fe-c2e7-452b-8bf0-88295f6ee140 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:53.664502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9697fe-c2e7-452b-8bf0-88295f6ee140 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:53.664782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9697fe-c2e7-452b-8bf0-88295f6ee140 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:53.672990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9697fe-c2e7-452b-8bf0-88295f6ee140 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:53.674498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9697fe-c2e7-452b-8bf0-88295f6ee140 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3f9697fe-c2e7-452b-8bf0-88295f6ee140 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:53.686424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:53.687357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:53.688307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:53.692235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 472/938] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:43:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:53.693682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:53.762016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef453-e112-483d-95b8-c74d43f48a84 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:53.762340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef453-e112-483d-95b8-c74d43f48a84 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:53.762561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ef453-e112-483d-95b8-c74d43f48a84 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:53.763890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 473/939] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:53.769143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf41c34-7cc1-4f80-965f-0346c154598c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:53.769421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf41c34-7cc1-4f80-965f-0346c154598c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:53.769689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf41c34-7cc1-4f80-965f-0346c154598c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:53.791363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2cf41c34-7cc1-4f80-965f-0346c154598c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:53.792909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2cf41c34-7cc1-4f80-965f-0346c154598c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:53.798011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:53.810155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1568d17-268a-4c99-ad69-08a6eca9cc2e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:53.856972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5a5a7-bb3a-424d-a904-2bf7b9ba4bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:53.857586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5a5a7-bb3a-424d-a904-2bf7b9ba4bbd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:53.857862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5a5a7-bb3a-424d-a904-2bf7b9ba4bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:53.858134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5a5a7-bb3a-424d-a904-2bf7b9ba4bbd None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:53.859157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5a5a7-bb3a-424d-a904-2bf7b9ba4bbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:53.859463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5a5a7-bb3a-424d-a904-2bf7b9ba4bbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:53.861190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5a5a7-bb3a-424d-a904-2bf7b9ba4bbd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:53.880268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 474/940] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:53.880851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 467/941] 10.222.0.50 () {56 vars in 1149 bytes} [Tue Aug 30 01:43:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:53.886220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf41c34-7cc1-4f80-965f-0346c154598c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:53.994276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf41c34-7cc1-4f80-965f-0346c154598c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.011905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:54.030966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.031617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.031897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.032169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.042856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.044350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.049747 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b316e2-7865-4563-918c-4c529518ea11 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.050036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b316e2-7865-4563-918c-4c529518ea11 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.050310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b316e2-7865-4563-918c-4c529518ea11 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.051892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 468/942] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:54.056800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111230-940f-4be1-bf0c-e5857c92964c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.057087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111230-940f-4be1-bf0c-e5857c92964c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.057362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111230-940f-4be1-bf0c-e5857c92964c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:54.060079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:54.061011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:54.061876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:54.061876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:54.061876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:54.065969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 475/943] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:54.067318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.076443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-31111230-940f-4be1-bf0c-e5857c92964c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:54.077786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-31111230-940f-4be1-bf0c-e5857c92964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:54.161762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 469/944] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:54.169096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31111230-940f-4be1-bf0c-e5857c92964c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.178530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5401b-0734-426f-9117-0275d323af68 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.216052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.217029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.217525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.217995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.218835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:54.219334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.220927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.234039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 476/945] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:54.240817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.274009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:54.281734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:54.300974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31111230-940f-4be1-bf0c-e5857c92964c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.338722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.339709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.340199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.340715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.341566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.343177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.346072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING keystone.server.flask.common [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 01:43:54.364708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 01:43:54.364708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 01:43:54.364708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [parameters: {'id': '096109047aa541dfab3caea61a4ef1de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 01:43:54.364708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 01:43:54.370444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bf077-bcb9-4220-97fa-184aba6fabd4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 096109047aa541dfab3caea61a4ef1de.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 096109047aa541dfab3caea61a4ef1de. Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:43:54.384170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 01:43:54.385150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Aug 30 01:43:54.386143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 01:43:54.387031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 01:43:54.387934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:43:54.388763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'id': '096109047aa541dfab3caea61a4ef1de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.389623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:54.390410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 096109047aa541dfab3caea61a4ef1de. Aug 30 01:43:54.391319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.399859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 470/946] 10.222.0.50 () {60 vars in 1140 bytes} [Tue Aug 30 01:43:54 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:54.401270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.408691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.409333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.409617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.409886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.418497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.420619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:54.436232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:54.437153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:54.438031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:54.440380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 477/947] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:54.441465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.515912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26facc-f35a-415d-b298-f2d34becf2dd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.534993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:54.548845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dcd77-cfd9-44ec-949e-5e4f5ea561e0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.550484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.551782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.551782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.551782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.555370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:54.559963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.562138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:54.576411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:54.577524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:54.578634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:54.579317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 471/948] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:54.579317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.594110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.594850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.595193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.595534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.596287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.597692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:54.606933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:54.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:54.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:54.610356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 478/949] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:54.611038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.701323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7d339-4b09-4e7d-802b-6d97e18dc21c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.717188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad616e5c-ee29-439c-9faa-9b54b9641891 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.735425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.736143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.736512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.737105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.737877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:54.738243 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.739746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.752634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.753339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.753685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.754025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.754754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:54.755127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.761160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 472/950] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:54.761160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.761367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.777623 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 479/951] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:54.778368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.797385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:54.835882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:54.894008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d6bc-c773-4e0e-853e-ae675daef18e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.901035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbd522-4b75-495f-b45d-608e12172472 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:54.932405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385883ec-0f47-4bf6-a199-3084377c6d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.933304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385883ec-0f47-4bf6-a199-3084377c6d40 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.933562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385883ec-0f47-4bf6-a199-3084377c6d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.934343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385883ec-0f47-4bf6-a199-3084377c6d40 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.934461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385883ec-0f47-4bf6-a199-3084377c6d40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:54.934740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385883ec-0f47-4bf6-a199-3084377c6d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.936099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385883ec-0f47-4bf6-a199-3084377c6d40 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.942226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 473/952] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:54.942548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177e22b-21b6-4489-9c7b-067461491074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:54.943253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177e22b-21b6-4489-9c7b-067461491074 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.943499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c676a-d190-47c0-82aa-7a8c9a2cf1cb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.943605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177e22b-21b6-4489-9c7b-067461491074 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.943782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c676a-d190-47c0-82aa-7a8c9a2cf1cb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.943871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177e22b-21b6-4489-9c7b-067461491074 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/b5ef7dccfc5e43be993e201a19730a1a/roles/3c74c79658db45408d0892a3de5fcb85` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.944058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c676a-d190-47c0-82aa-7a8c9a2cf1cb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.946466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177e22b-21b6-4489-9c7b-067461491074 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=b5ef7dccfc5e43be993e201a19730a1a, role_id=3c74c79658db45408d0892a3de5fcb85)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:54.947886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177e22b-21b6-4489-9c7b-067461491074 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:54.949551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 474/953] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:54.950307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c676a-d190-47c0-82aa-7a8c9a2cf1cb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:54.968859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 480/954] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:43:54 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/b5ef7dccfc5e43be993e201a19730a1a/roles/3c74c79658db45408d0892a3de5fcb85 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:43:54.971394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616459a-76f8-42d9-ac4a-c44a2d4deeee None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:54.971784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616459a-76f8-42d9-ac4a-c44a2d4deeee None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:54.972175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616459a-76f8-42d9-ac4a-c44a2d4deeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:54.994400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9616459a-76f8-42d9-ac4a-c44a2d4deeee None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:54.996463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9616459a-76f8-42d9-ac4a-c44a2d4deeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:55.077171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 481/955] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:55.079018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c676a-d190-47c0-82aa-7a8c9a2cf1cb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:55.104121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:55.120648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:55.121475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:55.121860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:55.122237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:55.123025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:55.123426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:55.124950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:55.130722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 475/956] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:55.136233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616459a-76f8-42d9-ac4a-c44a2d4deeee None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:55.140519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:55.240621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616459a-76f8-42d9-ac4a-c44a2d4deeee None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:55.277261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:55.278105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:55.278530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:55.278947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:55.279864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:55.281433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:55.312525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 482/957] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:43:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:43:55.405295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:55.517663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c79c-8495-424e-82ab-6b40a50688f5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:55.561298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:55.562179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:55.562644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:55.563062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:55.563798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:55.565368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:55.575551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:55.576888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:43:55.577954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:55.579922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 476/958] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:55.585906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:55.648519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:55.694185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1550d51-188e-4e2c-9af6-5ca50a1ac928 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:55.728705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3522282-c2ac-4dfd-bb48-47c8bd46ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:55.729455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522282-c2ac-4dfd-bb48-47c8bd46ecab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:55.729832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522282-c2ac-4dfd-bb48-47c8bd46ecab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:55.730258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522282-c2ac-4dfd-bb48-47c8bd46ecab None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:55.730991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3522282-c2ac-4dfd-bb48-47c8bd46ecab None admin] RBAC: Adding query filter params (name=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:55.731419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3522282-c2ac-4dfd-bb48-47c8bd46ecab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:55.732986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3522282-c2ac-4dfd-bb48-47c8bd46ecab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:55.738457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 483/959] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:55.748666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:55.797158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:55.858627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e724f-4b72-401b-a412-b68a3c4148d3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:55.894024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38e9cb-be9f-41dc-8824-c11e8aa75e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:55.894891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38e9cb-be9f-41dc-8824-c11e8aa75e67 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:55.895314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38e9cb-be9f-41dc-8824-c11e8aa75e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:55.895736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38e9cb-be9f-41dc-8824-c11e8aa75e67 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:55.896587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38e9cb-be9f-41dc-8824-c11e8aa75e67 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:55.897048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38e9cb-be9f-41dc-8824-c11e8aa75e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:55.898528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38e9cb-be9f-41dc-8824-c11e8aa75e67 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:55.903642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 477/960] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:43:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:56.873819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be1cd4-c4cf-4e1b-9fb0-317310ecdd1c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:56.874375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be1cd4-c4cf-4e1b-9fb0-317310ecdd1c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:56.874656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be1cd4-c4cf-4e1b-9fb0-317310ecdd1c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:56.876463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 484/961] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:56.881733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d70e2c-7fde-4ee2-8096-1b1cae7a308a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:56.882155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d70e2c-7fde-4ee2-8096-1b1cae7a308a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:56.882464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d70e2c-7fde-4ee2-8096-1b1cae7a308a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:56.902114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f4d70e2c-7fde-4ee2-8096-1b1cae7a308a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:56.903615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f4d70e2c-7fde-4ee2-8096-1b1cae7a308a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:56.917103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795eef5-f23d-4c08-98f3-a53bc31de0c9 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:56.917421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795eef5-f23d-4c08-98f3-a53bc31de0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:56.917647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795eef5-f23d-4c08-98f3-a53bc31de0c9 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:56.919238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 485/962] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:43:56.924379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ddbd-8b8f-4b94-9102-540f69cfc4f6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:56.924665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ddbd-8b8f-4b94-9102-540f69cfc4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:56.924941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9ddbd-8b8f-4b94-9102-540f69cfc4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:56.943054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-57f9ddbd-8b8f-4b94-9102-540f69cfc4f6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:56.944276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-57f9ddbd-8b8f-4b94-9102-540f69cfc4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:56.987962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 478/963] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:57.032477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 486/964] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:57.274121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d70e2c-7fde-4ee2-8096-1b1cae7a308a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:57.308931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9ddbd-8b8f-4b94-9102-540f69cfc4f6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:57.328533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:57.370990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:57.389909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d70e2c-7fde-4ee2-8096-1b1cae7a308a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:57.421396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9ddbd-8b8f-4b94-9102-540f69cfc4f6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:57.423604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:57.424205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.424515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.424784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.425415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:57.426835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:57.440865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 01:43:57.440865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 01:43:57.440865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [parameters: {'user_id': '36068e8f8c40479cac1e36888482844d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 01:43:57.440865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-87b4c353-fa91-4362-aa38-75b4987893fa None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 30 01:43:57.448046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:43:57.448966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 01:43:57.455367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 01:43:57.456308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 01:43:57.457214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 01:43:57.459837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'user_id': '36068e8f8c40479cac1e36888482844d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:57.462088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 01:43:57.464054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 479/965] 10.222.0.50 () {60 vars in 1131 bytes} [Tue Aug 30 01:43:57 2022] POST /identity/v3/users => generated 160 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff1f8d-798b-470a-bcf5-42546aa6785d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff1f8d-798b-470a-bcf5-42546aa6785d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.465996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff1f8d-798b-470a-bcf5-42546aa6785d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.467331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 480/966] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:57.467331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff1f8d-798b-470a-bcf5-42546aa6785d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:57.467567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b3b00-5c7a-4f00-8dd5-1367335aa76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:57.467567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b3b00-5c7a-4f00-8dd5-1367335aa76f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.467567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b3b00-5c7a-4f00-8dd5-1367335aa76f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.467567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b3b00-5c7a-4f00-8dd5-1367335aa76f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.478308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b3b00-5c7a-4f00-8dd5-1367335aa76f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-234b3b00-5c7a-4f00-8dd5-1367335aa76f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:43:57.483159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:57.488760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 487/967] 10.222.0.50 () {56 vars in 1120 bytes} [Tue Aug 30 01:43:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:57.489586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb55066-5b84-47c9-b769-8ff65399f965 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.491037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb55066-5b84-47c9-b769-8ff65399f965 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.491037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb55066-5b84-47c9-b769-8ff65399f965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.505098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-eeb55066-5b84-47c9-b769-8ff65399f965 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:57.506461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eeb55066-5b84-47c9-b769-8ff65399f965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:57.569823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:57.574802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff1f8d-798b-470a-bcf5-42546aa6785d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:57.580289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 488/968] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:43:57.580289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb55066-5b84-47c9-b769-8ff65399f965 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:57.616202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:57.617080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.617422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.617756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.625326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:57.626981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:57.634860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:57.640866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:57.641989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 01:43:57.643159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:57.648896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 481/969] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:43:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:57.650150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:57.702221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb55066-5b84-47c9-b769-8ff65399f965 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:57.735894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:57.736851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.737198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.737537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.738298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:57.738665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:57.740689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:57.753829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 489/970] 10.222.0.50 () {56 vars in 1115 bytes} [Tue Aug 30 01:43:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:57.755102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:57.764310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce76ffc-2143-4393-9e0c-65e2f2e65b5b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:57.801140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd7ac-e698-426e-ba58-b3b44096971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:57.801921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd7ac-e698-426e-ba58-b3b44096971b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.802298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd7ac-e698-426e-ba58-b3b44096971b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.802724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd7ac-e698-426e-ba58-b3b44096971b None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.803517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd7ac-e698-426e-ba58-b3b44096971b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:57.803913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd7ac-e698-426e-ba58-b3b44096971b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:57.805452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd7ac-e698-426e-ba58-b3b44096971b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:57.848355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 482/971] 10.222.0.50 () {56 vars in 1101 bytes} [Tue Aug 30 01:43:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:43:57.848355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e561-2f39-4c79-80e1-902a1214152b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.848355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e561-2f39-4c79-80e1-902a1214152b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.848355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e561-2f39-4c79-80e1-902a1214152b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.848355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 483/972] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:43:57.851678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd624e-f315-484e-a123-ee243c397bdb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.851970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd624e-f315-484e-a123-ee243c397bdb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.852267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd624e-f315-484e-a123-ee243c397bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.872726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-04cd624e-f315-484e-a123-ee243c397bdb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:43:57.874318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-04cd624e-f315-484e-a123-ee243c397bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:43:57.887986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbb40c-f483-40e2-879e-a9e0b61d2274 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:57.889702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:57.892354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:57.928710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:57.929537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:57.930305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:57.930685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:57.944160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:57.945700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:57.957152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 484/973] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:43:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:43:57.958089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd624e-f315-484e-a123-ee243c397bdb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:57.973589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:57.974677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:57.975545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:57.975545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:57.975545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:57.975545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 490/974] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:57.981261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.085256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd624e-f315-484e-a123-ee243c397bdb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.088989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad73631-dd82-4f40-bddc-d46af4e5eeb1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.128652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.129597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.129979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.130354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.131575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.133113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.133894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.134660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.135346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.135763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.136896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:58.137196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.138696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:58.144227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:58.145436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:43:58.146490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:58.151101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 485/975] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:43:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:58.153509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 491/976] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:58.155160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.158296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.174249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:58.176535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:58.259305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0dd5-cfbb-4502-a780-0c472bf8ad11 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.264862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefabe0-1dd4-417e-b932-690c3a665eaa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.291910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.292585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.292864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.293142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.298685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.299289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.299562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.299831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.300131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.300478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:58.300778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.302121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.303018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.307563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 486/977] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:58.312884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:43:58.314752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:58.315808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 492/978] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:43:58.316956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.392785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:58.392955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:58.415179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dcc2f-9331-43ec-ba2d-41274ec6f4b7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.415430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f3304-0c33-4343-8c95-e55024d0aca5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.448252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.448538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.448872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.449133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.449227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.449410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.449500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.449680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.450069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.450302 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:58.450603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.451421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.451954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.456828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 493/979] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:58.457612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.468351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 487/980] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:43:58 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:43:58.468351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.572150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03106af-e893-4226-b92c-20ddb1920791 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.607296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908418a9-35b8-42dd-9d8e-7249b6213f08 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.613058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.613671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.613947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.614214 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.614965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:58.615272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.616742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.626292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:58.635549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 494/981] 10.222.0.50 () {56 vars in 1149 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:58.641070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.647030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:58.652651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.653692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.654621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.655268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.655984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] RBAC: Adding query filter params (interface=admin, service_id=fccf870e8f3649538dd02953fe175cd9, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:58.656321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.657691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.663015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 488/982] 10.222.0.50 () {56 vars in 1249 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/endpoints?service_id=fccf870e8f3649538dd02953fe175cd9&interface=admin®ion_id=RegionOne => generated 193 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:58.668469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:58.749222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9cdd7-332f-40b4-804c-33e685205064 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.775926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d5070-74f2-42a2-873f-355b37ba5ead None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:58.786261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.787191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.787712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.787898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.811729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551786d-d27c-4529-a414-d014a97157da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:58.812507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551786d-d27c-4529-a414-d014a97157da None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:58.812789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551786d-d27c-4529-a414-d014a97157da None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:58.813361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551786d-d27c-4529-a414-d014a97157da None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:58.814227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551786d-d27c-4529-a414-d014a97157da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.816036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551786d-d27c-4529-a414-d014a97157da None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.832322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 489/983] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:43:58.857516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:58.862477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:58.863959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:58.973847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 495/984] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:43:58.979724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551786d-d27c-4529-a414-d014a97157da None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:59.041770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:59.083724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551786d-d27c-4529-a414-d014a97157da None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:59.117554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:59.118488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:59.118942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:59.119396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:59.130053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:59.132272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:43:59.146550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:59.147499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:43:59.148428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:43:59.148428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:43:59.148428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:59.149632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 490/985] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:59.155283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:59.252199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:59.257627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77dd2-6877-4a24-ba82-da41b99ae5d7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:59.298816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:59.299432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:59.299709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:59.299986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:59.300713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:59.301020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:59.302417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:59.314873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 496/986] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:59.320190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:59.422727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f455d-0334-4255-8462-bf4a51746a90 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:59.435755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:43:59.457319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:59.457945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:59.458220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:59.458487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:59.465073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:59.466500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:43:59.478038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:43:59.479269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:43:59.480146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 491/987] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:43:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:43:59.484939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:59.584538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0fae4-af50-4e07-98bf-63d91f2ae7c9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:59.620313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a87820-3ce3-4bd6-811c-c8ba3883bc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:59.620915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a87820-3ce3-4bd6-811c-c8ba3883bc3f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:59.621193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a87820-3ce3-4bd6-811c-c8ba3883bc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:59.621464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a87820-3ce3-4bd6-811c-c8ba3883bc3f None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:59.622146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a87820-3ce3-4bd6-811c-c8ba3883bc3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:59.622449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a87820-3ce3-4bd6-811c-c8ba3883bc3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:59.623938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a87820-3ce3-4bd6-811c-c8ba3883bc3f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:59.635656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 497/988] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:43:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:43:59.641433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:43:59.756369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec7e3e-5453-4170-a285-0cb073499d74 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:43:59.790884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d1c43-8b52-4b5f-8b37-51a012cb02b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:43:59.791486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d1c43-8b52-4b5f-8b37-51a012cb02b5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:43:59.791857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d1c43-8b52-4b5f-8b37-51a012cb02b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:43:59.792115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d1c43-8b52-4b5f-8b37-51a012cb02b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:43:59.792859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d1c43-8b52-4b5f-8b37-51a012cb02b5 None admin] RBAC: Adding query filter params (role.id=3c74c79658db45408d0892a3de5fcb85, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=b5ef7dccfc5e43be993e201a19730a1a) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:43:59.793167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d1c43-8b52-4b5f-8b37-51a012cb02b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:43:59.794652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d1c43-8b52-4b5f-8b37-51a012cb02b5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:43:59.800465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 492/989] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:43:59 2022] GET /identity/v3/role_assignments?user.id=b5ef7dccfc5e43be993e201a19730a1a&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3c74c79658db45408d0892a3de5fcb85 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:00.431937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3175f-95ad-4e73-8766-dc1adfd96194 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:00.432489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3175f-95ad-4e73-8766-dc1adfd96194 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:00.432489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3175f-95ad-4e73-8766-dc1adfd96194 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:00.434046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 498/990] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:00.439366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231e3d8-3b39-410e-ba8c-ada41228f234 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:00.439761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231e3d8-3b39-410e-ba8c-ada41228f234 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:00.439937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231e3d8-3b39-410e-ba8c-ada41228f234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:00.459348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1231e3d8-3b39-410e-ba8c-ada41228f234 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:00.460540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1231e3d8-3b39-410e-ba8c-ada41228f234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:00.490164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:00.528752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 493/991] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:00.756581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7662c0d-e4d0-4134-bcb2-a6b55d653da5 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:00.756883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7662c0d-e4d0-4134-bcb2-a6b55d653da5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:00.757142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7662c0d-e4d0-4134-bcb2-a6b55d653da5 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:00.758619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 499/992] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:00.763504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc72f-800a-4b62-98bc-63df7fbf8fad None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:00.764523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc72f-800a-4b62-98bc-63df7fbf8fad None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:00.764805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc72f-800a-4b62-98bc-63df7fbf8fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:00.784005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e6ccc72f-800a-4b62-98bc-63df7fbf8fad None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:00.785228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6ccc72f-800a-4b62-98bc-63df7fbf8fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:00.795703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7662c0d-e4d0-4134-bcb2-a6b55d653da5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:00.816096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:00.861411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 494/993] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:00.911038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7662c0d-e4d0-4134-bcb2-a6b55d653da5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:00.944445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:00.945065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:00.945337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:00.945604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:00.946176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:00.947529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:00.956422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:00.957549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:44:00.958385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:00.960158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 500/994] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:44:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:00.963361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc72f-800a-4b62-98bc-63df7fbf8fad None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.055863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:01.062817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc72f-800a-4b62-98bc-63df7fbf8fad None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.096568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.097168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.097440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.097706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.098318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:01.098615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.099977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.110841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 495/995] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:44:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:01.110998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.134716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.201796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:01.221553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d995-9069-4d26-a637-7a70ab6c31b9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.246581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087af35-98be-49bc-acd4-97ef92205ed3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.257874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.258493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.258775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.259054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.259688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:01.259992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.261402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.271065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 501/996] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:44:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:01.273203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.287254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.287254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.287254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.287254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.287694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.288450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:01.297797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:01.298725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:01.300308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 496/997] 10.222.0.50 () {56 vars in 1126 bytes} [Tue Aug 30 01:44:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:01.305286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.332115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:01.393703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14a3e3-f1d4-412d-b1c7-67ffbbcae89c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.417210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d808b2-044e-4ee0-923f-65481366cf0d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.434012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.434729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.435015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.435291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.435998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] RBAC: Adding query filter params (interface=public, service_id=4923249a4337433aa47f8ab32d6ac2d6, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:01.436335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.437699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.442868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 502/998] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:44:01 2022] GET /identity/v3/endpoints?service_id=4923249a4337433aa47f8ab32d6ac2d6&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:01.448687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.453106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.453718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.453998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.454272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.454901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:01.455214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.456594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.462097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 497/999] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:44:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:01.465181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:01.467910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.560429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:01.565347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6f821-690d-4cdb-9ee1-b8aa11f37da5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.577975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6881-813b-4d33-aa66-db2268cbb0f3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.603072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.603611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.603889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.604168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.604814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.606225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.612870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 503/1000] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:44:01 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:44:01.617671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e4903-1708-4e48-a4c0-af2a975534c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.618306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4903-1708-4e48-a4c0-af2a975534c7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.618588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4903-1708-4e48-a4c0-af2a975534c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.618866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e4903-1708-4e48-a4c0-af2a975534c7 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.619512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e4903-1708-4e48-a4c0-af2a975534c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:01.619819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e4903-1708-4e48-a4c0-af2a975534c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.621294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e4903-1708-4e48-a4c0-af2a975534c7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.632906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 498/1001] 10.222.0.50 () {56 vars in 1122 bytes} [Tue Aug 30 01:44:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:01.633987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:01.718064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:01.724220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc284a5-e01a-4f1d-8cb2-80501d14e9b9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.724770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc284a5-e01a-4f1d-8cb2-80501d14e9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.725256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc284a5-e01a-4f1d-8cb2-80501d14e9b9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.727668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 499/1002] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:44:01.733516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15fecf-0d30-4ca1-a7bc-17fc1e4cb8bf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.733813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15fecf-0d30-4ca1-a7bc-17fc1e4cb8bf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.734069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15fecf-0d30-4ca1-a7bc-17fc1e4cb8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.749869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f82fd-04df-4dff-bcbd-48af0bfe8634 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:01.754398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bb15fecf-0d30-4ca1-a7bc-17fc1e4cb8bf None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:01.756015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb15fecf-0d30-4ca1-a7bc-17fc1e4cb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:01.793051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:01.793658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:01.793941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:01.794303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:01.794873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:01.796235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:01.813659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 504/1003] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:44:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:44:01.843831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 500/1004] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:02.108450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:02.210553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba8d51-1a0c-44b0-9c8a-fffd194f5829 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:02.243996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:02.244649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:02.244932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:02.245204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] PATH_INFO: `/v3/services/image` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:02.245782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:02.247144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:02.248958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:02.257861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:02.259143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:02.260227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:02.264312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 505/1005] 10.222.0.50 () {56 vars in 1105 bytes} [Tue Aug 30 01:44:02 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:02.267580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15fecf-0d30-4ca1-a7bc-17fc1e4cb8bf None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:02.353227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:02.375635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15fecf-0d30-4ca1-a7bc-17fc1e4cb8bf None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:02.410439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce34a8-c34c-44ed-b017-d2fda0878e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:02.411042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce34a8-c34c-44ed-b017-d2fda0878e5a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:02.411323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce34a8-c34c-44ed-b017-d2fda0878e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:02.411594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce34a8-c34c-44ed-b017-d2fda0878e5a None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:02.412267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce34a8-c34c-44ed-b017-d2fda0878e5a None admin] RBAC: Adding query filter params (name=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:02.412570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce34a8-c34c-44ed-b017-d2fda0878e5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:02.413914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce34a8-c34c-44ed-b017-d2fda0878e5a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:02.418689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 501/1006] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:02.424279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:02.523761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9775ea-0af0-49f6-9414-b0bd34dedd2b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:02.557542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d67340-67c6-46f9-bd1c-5b599c3ac381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:02.558149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d67340-67c6-46f9-bd1c-5b599c3ac381 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:02.558425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d67340-67c6-46f9-bd1c-5b599c3ac381 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:02.558697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d67340-67c6-46f9-bd1c-5b599c3ac381 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:02.559324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d67340-67c6-46f9-bd1c-5b599c3ac381 None admin] RBAC: Adding query filter params (type=image) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:02.559626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d67340-67c6-46f9-bd1c-5b599c3ac381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:02.561060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d67340-67c6-46f9-bd1c-5b599c3ac381 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:02.565546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 506/1007] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:03.604054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b005e1-0c45-48d0-bc2c-690ac612845a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:03.604591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b005e1-0c45-48d0-bc2c-690ac612845a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:03.604591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b005e1-0c45-48d0-bc2c-690ac612845a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:03.606062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 502/1008] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:03.611198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054399ed-d242-485a-b335-100bf4dca265 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:03.611955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054399ed-d242-485a-b335-100bf4dca265 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:03.612037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054399ed-d242-485a-b335-100bf4dca265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:03.630962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-054399ed-d242-485a-b335-100bf4dca265 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:03.632161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-054399ed-d242-485a-b335-100bf4dca265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:03.690887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:03.702780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 507/1009] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:03.777654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dc883-7608-469b-a6ac-1fab4d167bcd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:03.777975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dc883-7608-469b-a6ac-1fab4d167bcd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:03.778216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dc883-7608-469b-a6ac-1fab4d167bcd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:03.779688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 503/1010] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:03.784664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae13ff-2371-420d-bf60-2593431d5cbd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:03.784952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae13ff-2371-420d-bf60-2593431d5cbd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:03.785228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae13ff-2371-420d-bf60-2593431d5cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:03.803832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-20ae13ff-2371-420d-bf60-2593431d5cbd None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:03.805026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-20ae13ff-2371-420d-bf60-2593431d5cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:03.873695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 508/1011] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:03.977574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dc883-7608-469b-a6ac-1fab4d167bcd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:04.080650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dc883-7608-469b-a6ac-1fab4d167bcd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:04.116004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:04.116663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.116945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.117222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.117802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:04.119156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:04.127945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:04.128950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 01:44:04.129816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:04.132586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 504/1012] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:44:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:04.135350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae13ff-2371-420d-bf60-2593431d5cbd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:04.157534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:04.167108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:04.238959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae13ff-2371-420d-bf60-2593431d5cbd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:04.263161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:04.266969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7359157-b660-481e-86fe-099b43a52274 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:04.275897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:04.276540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.276824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.277096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.277911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:04.278020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:04.279388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:04.284283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 509/1013] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:44:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:04.289817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:04.308998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:04.309626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.309903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.310172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.311276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:04.312728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:04.314519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 505/1014] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:04.320619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:04.395830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f67a1b-498c-499f-a258-e3406387c501 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:04.413017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:04.431399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:04.432012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.432322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.432599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.433227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:04.433531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:04.434897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:04.439792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 510/1015] 10.222.0.50 () {56 vars in 1112 bytes} [Tue Aug 30 01:44:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:04.440314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b73b8e-f5e9-4bf3-94de-527ceee7f6e4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:04.445196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:04.474061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c121e-8530-4a06-95b5-d45b741e78ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:04.475218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c121e-8530-4a06-95b5-d45b741e78ae None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.475492 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c121e-8530-4a06-95b5-d45b741e78ae None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.475769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c121e-8530-4a06-95b5-d45b741e78ae None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.476426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c121e-8530-4a06-95b5-d45b741e78ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:04.477833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c121e-8530-4a06-95b5-d45b741e78ae None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:04.494265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:04.505084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 506/1016] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:44:04 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:44:04.506315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d6d1-da82-4ba6-a983-467bfcf13822 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.506600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d6d1-da82-4ba6-a983-467bfcf13822 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.506924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d6d1-da82-4ba6-a983-467bfcf13822 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.508075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 507/1017] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:04.513731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46847303-0d79-4136-9360-6a34c94f804b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.514006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46847303-0d79-4136-9360-6a34c94f804b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.514277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46847303-0d79-4136-9360-6a34c94f804b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.533685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-46847303-0d79-4136-9360-6a34c94f804b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:04.534912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46847303-0d79-4136-9360-6a34c94f804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:04.562583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d422-7d1b-4bfe-bf0d-66f48fd40c64 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:04.601960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcda355-d863-455b-aaa9-8ed6ee076c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:04.602580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda355-d863-455b-aaa9-8ed6ee076c0b None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:04.602856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda355-d863-455b-aaa9-8ed6ee076c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:04.603126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda355-d863-455b-aaa9-8ed6ee076c0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:04.603801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcda355-d863-455b-aaa9-8ed6ee076c0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:04.605219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcda355-d863-455b-aaa9-8ed6ee076c0b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:04.615059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 508/1018] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:04.628309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 511/1019] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:44:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:44:04.881718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46847303-0d79-4136-9360-6a34c94f804b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:04.946123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:04.994974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46847303-0d79-4136-9360-6a34c94f804b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:05.035232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74e499-b1d5-4c1d-94c0-f6eff0a9b5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:05.035894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74e499-b1d5-4c1d-94c0-f6eff0a9b5e2 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:05.036172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74e499-b1d5-4c1d-94c0-f6eff0a9b5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:05.036537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74e499-b1d5-4c1d-94c0-f6eff0a9b5e2 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:05.037278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74e499-b1d5-4c1d-94c0-f6eff0a9b5e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:05.038651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74e499-b1d5-4c1d-94c0-f6eff0a9b5e2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:05.060294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 509/1020] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:44:04 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:06.456123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac7595-e2b7-41b1-a52f-e9b8d0cc4ff4 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:06.456708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac7595-e2b7-41b1-a52f-e9b8d0cc4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:06.456708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac7595-e2b7-41b1-a52f-e9b8d0cc4ff4 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:06.458060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 512/1021] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:06.463589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091cb58-32b3-4293-9659-a2053ae88330 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:06.463968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091cb58-32b3-4293-9659-a2053ae88330 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:06.464141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091cb58-32b3-4293-9659-a2053ae88330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:06.482732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3091cb58-32b3-4293-9659-a2053ae88330 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:06.483904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3091cb58-32b3-4293-9659-a2053ae88330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:06.557079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 510/1022] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:06.620700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86acc82-0527-43e8-9709-7b94a3b2186a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:06.621007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86acc82-0527-43e8-9709-7b94a3b2186a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:06.621253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86acc82-0527-43e8-9709-7b94a3b2186a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:06.622734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 513/1023] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:06.627665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f9245-c2fd-4940-9b76-0e59c1c018fb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:06.627955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f9245-c2fd-4940-9b76-0e59c1c018fb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:06.628235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f9245-c2fd-4940-9b76-0e59c1c018fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:06.646815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b95f9245-c2fd-4940-9b76-0e59c1c018fb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:06.647996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b95f9245-c2fd-4940-9b76-0e59c1c018fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:06.677461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:06.716573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 511/1024] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:06.826702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86acc82-0527-43e8-9709-7b94a3b2186a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:06.935257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86acc82-0527-43e8-9709-7b94a3b2186a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:06.942046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:06.950515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08884c1-49d4-4347-81bc-a9a19c76b0ed None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:06.950815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08884c1-49d4-4347-81bc-a9a19c76b0ed None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:06.951069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08884c1-49d4-4347-81bc-a9a19c76b0ed None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:06.952680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 512/1025] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:06.957337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f033ea-54e8-4662-9f0d-c3fa1e2b5c5e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:06.957618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f033ea-54e8-4662-9f0d-c3fa1e2b5c5e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:06.957898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f033ea-54e8-4662-9f0d-c3fa1e2b5c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:06.973604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:06.974262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:06.974545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:06.974821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:06.975925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:06.977430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:06.977918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d6f033ea-54e8-4662-9f0d-c3fa1e2b5c5e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:06.979002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 514/1026] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:06.979508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6f033ea-54e8-4662-9f0d-c3fa1e2b5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:06.981772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.045433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 513/1027] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:07.046237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f033ea-54e8-4662-9f0d-c3fa1e2b5c5e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.088158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04c2b5-2da5-49c0-9743-7c363e41c729 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.120679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.121304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.121578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.121847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.132095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:07.136928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:07.137851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:07.141213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 515/1028] 10.222.0.50 () {56 vars in 1120 bytes} [Tue Aug 30 01:44:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:07.143762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.154489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:07.160978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f033ea-54e8-4662-9f0d-c3fa1e2b5c5e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.184717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:07.206861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.207764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.208186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.208661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.209661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.211903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.236288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 514/1029] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:44:07 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:07.272153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b0010-a28b-411a-85ef-a0a70ee6b55b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.317819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.319728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.320552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.320979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.321504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.322358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:07.322664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.324062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.333597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 516/1030] 10.222.0.50 () {56 vars in 1115 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:07.339653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.438794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeec18-4d2c-4fef-9906-dc21cf27f195 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.446181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:07.454652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18db2-0570-47cb-b6ad-e3bd4b461a91 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.480005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.480871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.481149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.481719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] PATH_INFO: `/v3/services/image` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.482565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.484400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.489569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.490726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.491005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.491277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:07.495170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:07.496122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:07.497332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 515/1031] 10.222.0.50 () {56 vars in 1105 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:07.502207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.504599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.507644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.522655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:07.525441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:07.526561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:44:07.529630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:07.529630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:07.529630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:07.529630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 517/1032] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:07.532646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.613078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386b278-70d9-4c19-a175-ae14bebf6efb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.643806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525c36f-a8cb-4941-bbec-29cad1aeacc4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.647110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.647712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.647984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.648269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.648894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] RBAC: Adding query filter params (name=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:07.649192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.650535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.655605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 516/1033] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:07.660907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.680206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.680831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.681106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.681377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.682004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:07.682305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.683667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.695928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 518/1034] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:07.701423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.712376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:07.748406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:07.768208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7baa9-32dd-42d9-82e5-dcf025e4e15d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.806358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.807029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.807305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.807582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.808229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] RBAC: Adding query filter params (type=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:07.808556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.809913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.810807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3488b-f213-4a73-a77d-e31199887a0c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.814812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 517/1035] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:07.820972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.848553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.849677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.849958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.850226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.857215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.858648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:44:07.870945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:07.872075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:44:07.873319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:07.876778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 519/1036] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:07.877650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:07.942308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:07.947871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb65b3-1008-4f19-90b1-4557e19ffc0b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.978175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d50f2-c2e8-4304-8787-c54738a275db None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:07.986491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:07.987347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:07.987631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:07.987906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:07.988635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] RBAC: Adding query filter params (interface=public, service_id=6097647ff9e14109be7930cc7df9dcd6, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:07.988942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:07.990326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:07.995283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 518/1037] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/endpoints?service_id=6097647ff9e14109be7930cc7df9dcd6&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:08.000724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:08.013698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:08.023301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:08.023911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:08.024189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:08.024490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:08.025225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:08.025525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:08.026948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:08.053251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 520/1038] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:08.054500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:08.111278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab8174-616e-4db7-9dcb-e63459dfa59e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:08.146521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:08.147109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:08.147354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:08.147621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:08.148215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:08.149589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:08.156270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 519/1039] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:44:07 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:44:08.163247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961684d-87c6-4727-b648-b0d2ae894cb8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:08.192633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:08.202977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:08.203603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:08.203876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:08.204143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:08.281172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:08.282645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:08.389999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 521/1040] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:44:08.395593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:08.470391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:08.497600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f972f44-97a4-4643-b726-2118a252fdf3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:08.529957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:08.530576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:08.530841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:08.531114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:08.541771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:08.543191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:08.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:08.558321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:44:08.559205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:08.559205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:08.559205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:08.559205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 520/1041] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:08.564351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:08.591372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:08.665314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2107c0-70e8-4231-82aa-3b455a2057b8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:08.698543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:08.699141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:08.699414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:08.699681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:08.700406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:08.700707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:08.702060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:08.718136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 522/1042] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:08.719885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:08.819134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3d0db-2464-42b9-93fb-48f944ff9d72 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:08.844690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:08.856608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:08.857237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:08.857511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:08.857778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:08.864749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:08.866229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:44:08.877263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:08.878141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:08.883819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 521/1043] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:44:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:08.885786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:08.974978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:08.991047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486c5ad-7198-44d1-8573-bab156e7f3ce None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:09.024257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd0e09-ebfc-43c0-b44d-f4020fe46dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:09.024865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd0e09-ebfc-43c0-b44d-f4020fe46dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:09.025139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd0e09-ebfc-43c0-b44d-f4020fe46dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:09.025406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd0e09-ebfc-43c0-b44d-f4020fe46dc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:09.026084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd0e09-ebfc-43c0-b44d-f4020fe46dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:09.026384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd0e09-ebfc-43c0-b44d-f4020fe46dc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:09.027792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd0e09-ebfc-43c0-b44d-f4020fe46dc0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:09.040125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 523/1044] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:44:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:09.045887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:09.145695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd272d4-6a9e-45aa-bf1a-b99debcc4c02 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:09.178931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1fb84-7e73-47f5-9911-627306204c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:09.179533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fb84-7e73-47f5-9911-627306204c76 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:09.179807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fb84-7e73-47f5-9911-627306204c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:09.180075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fb84-7e73-47f5-9911-627306204c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:09.180822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1fb84-7e73-47f5-9911-627306204c76 None admin] RBAC: Adding query filter params (role.id=3c74c79658db45408d0892a3de5fcb85, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=95b365b4794c4edebe306e09336c42f3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:09.181130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1fb84-7e73-47f5-9911-627306204c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:09.182536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1fb84-7e73-47f5-9911-627306204c76 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:09.188088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 522/1045] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:09 2022] GET /identity/v3/role_assignments?user.id=95b365b4794c4edebe306e09336c42f3&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3c74c79658db45408d0892a3de5fcb85 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:09.228310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94354cd5-e0ee-4915-9c89-84a4dd3fd4f8 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:09.228650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94354cd5-e0ee-4915-9c89-84a4dd3fd4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:09.228845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94354cd5-e0ee-4915-9c89-84a4dd3fd4f8 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:09.230461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 524/1046] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:09.236273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793055f-2dd5-4d82-945c-33f7a53783e7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:09.236579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793055f-2dd5-4d82-945c-33f7a53783e7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:09.237108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793055f-2dd5-4d82-945c-33f7a53783e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:09.256875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0793055f-2dd5-4d82-945c-33f7a53783e7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:09.258091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0793055f-2dd5-4d82-945c-33f7a53783e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:09.314813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:09.342498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 523/1047] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:09.608783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94354cd5-e0ee-4915-9c89-84a4dd3fd4f8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:09.711275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94354cd5-e0ee-4915-9c89-84a4dd3fd4f8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:09.745292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:09.745922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:09.746205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:09.746479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:09.756975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:09.761801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:09.762731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:09.763626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 525/1048] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:09.770433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793055f-2dd5-4d82-945c-33f7a53783e7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:09.872274 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793055f-2dd5-4d82-945c-33f7a53783e7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:09.904815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7d1a7-c884-4666-98bc-4bad2ae3d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:09.905424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d1a7-c884-4666-98bc-4bad2ae3d714 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:09.905701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d1a7-c884-4666-98bc-4bad2ae3d714 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:09.905971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d1a7-c884-4666-98bc-4bad2ae3d714 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:09.906625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7d1a7-c884-4666-98bc-4bad2ae3d714 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:09.906927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7d1a7-c884-4666-98bc-4bad2ae3d714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:09.908308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7d1a7-c884-4666-98bc-4bad2ae3d714 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:09.921740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 524/1049] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:09.923148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:09.935841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:10.026072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb0ebe-5e7e-4061-91fa-c98253fd4981 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:10.054145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cae72-2bd5-4edc-8be0-a19941deb5ba None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.054433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cae72-2bd5-4edc-8be0-a19941deb5ba None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.054700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cae72-2bd5-4edc-8be0-a19941deb5ba None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.056161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 525/1050] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:44:10.060442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:10.060693 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca1f4c-b7c9-4c3f-8ef2-51b95f548705 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.060952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca1f4c-b7c9-4c3f-8ef2-51b95f548705 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.061117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.061210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca1f4c-b7c9-4c3f-8ef2-51b95f548705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.061352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.061609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] PATH_INFO: `/v3/users/ce282db7581147d092977648f82cea14` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.074914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] RBAC: Authorizing `identity:get_user(user_id=ce282db7581147d092977648f82cea14)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:10.076504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:10.082847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 526/1051] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:09 2022] GET /identity/v3/users/ce282db7581147d092977648f82cea14 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:10.082967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-74ca1f4c-b7c9-4c3f-8ef2-51b95f548705 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:10.082967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-74ca1f4c-b7c9-4c3f-8ef2-51b95f548705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:10.083729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:10.118461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:10.151322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:10.156434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 526/1052] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:10.191116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53408e94-c4a9-4e2a-94d5-2bb035a6369c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:10.232987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:10.233955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.234377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.234791 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.322399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:10.323858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:10.385286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:10.425230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca1f4c-b7c9-4c3f-8ef2-51b95f548705 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:10.430137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 527/1053] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:44:10.431145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:10.529003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca1f4c-b7c9-4c3f-8ef2-51b95f548705 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:10.542203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf97bc1-e669-4596-b8f0-3804ccf4f2fe None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:10.566227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:10.566874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.567152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.567573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.568878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:10.570473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:10.581705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:10.582679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:10.583596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 527/1054] 10.222.0.50 () {56 vars in 1105 bytes} [Tue Aug 30 01:44:10 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:10.592235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:10.594332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:10.594985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.595294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.595551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.596742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbd2b62613b424a90f1185e6cb9b5d7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:10.598195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:10.599916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 528/1055] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:10 2022] GET /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:10.606410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:10.638063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:10.639916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:10.710544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfc5e9-c444-4cce-8a7c-475ab1a36fb8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:10.718489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dfb4b-950a-4185-8738-37936e9c4e20 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:10.746661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:10.747273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.747568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.747828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.748480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:10.748781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:10.750140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:10.760115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 528/1056] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:10.761161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:10.761289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:10.761775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.762055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.762328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.763056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7, user.id=ce282db7581147d092977648f82cea14) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:10.763370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:10.764844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:10.770553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 529/1057] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:10 2022] GET /identity/v3/role_assignments?user.id=ce282db7581147d092977648f82cea14&scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:10.877522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:10.882996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51031540-2dce-4e94-9ae2-cb39a2a18ca2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:10.921859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732f4c-4ff4-496f-8b21-7eda6b179256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:10.922475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732f4c-4ff4-496f-8b21-7eda6b179256 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:10.922751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732f4c-4ff4-496f-8b21-7eda6b179256 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:10.923025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732f4c-4ff4-496f-8b21-7eda6b179256 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:10.923656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82732f4c-4ff4-496f-8b21-7eda6b179256 None admin] RBAC: Adding query filter params (type=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:10.923959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82732f4c-4ff4-496f-8b21-7eda6b179256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:10.925362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82732f4c-4ff4-496f-8b21-7eda6b179256 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:10.931401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 529/1058] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:10.936838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:11.033046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:11.040051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb740-2ac0-44d0-84f8-3a67c4d71685 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:11.067529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0f5c2-5f17-4083-8984-20400ac32750 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:11.068457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0f5c2-5f17-4083-8984-20400ac32750 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:11.068671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0f5c2-5f17-4083-8984-20400ac32750 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:11.069993 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 530/1059] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:44:11.074700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:11.074838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf5a1d-4026-4a3a-a357-96bdc4652e0c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:11.075044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf5a1d-4026-4a3a-a357-96bdc4652e0c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:11.075306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:11.075403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf5a1d-4026-4a3a-a357-96bdc4652e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:11.075583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:11.075856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:11.076557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:11.077921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:11.093449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8daf5a1d-4026-4a3a-a357-96bdc4652e0c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:11.094853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8daf5a1d-4026-4a3a-a357-96bdc4652e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:11.099953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 530/1060] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:44:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:44:11.177108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 531/1061] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:11.440333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:11.540364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b6737-80e1-4bfd-8c15-27aa6616ccba None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:11.576650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:11.577363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:11.577732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:11.578094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:11.588595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:11.593416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:11.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:44:11.595522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:11.597310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 531/1062] 10.222.0.50 () {56 vars in 1120 bytes} [Tue Aug 30 01:44:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:11.603496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf5a1d-4026-4a3a-a357-96bdc4652e0c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:11.692570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:11.702713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf5a1d-4026-4a3a-a357-96bdc4652e0c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:11.737821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:11.738428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:11.738703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:11.738973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:11.739626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:11.739956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:11.741339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:11.753760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 532/1063] 10.222.0.50 () {56 vars in 1115 bytes} [Tue Aug 30 01:44:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:11.755405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:11.770703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:11.859711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a65dfe-c2a8-4c2c-806a-dec255487c65 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:11.892102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:11.892746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:11.893019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:11.893284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:11.904361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:11.906186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:11.920780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:11.921703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:44:11.927433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:11.927433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:11.927433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:11.927433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 532/1064] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:11.927673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:12.024900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420c86a-cecc-47de-831f-3a805bacab6d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:12.057198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:12.057791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.058065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.058333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.058956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:12.059257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:12.060690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:12.073213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 533/1065] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:12.078621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:12.140980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:12.180571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163494-8a76-4a40-91e9-dbf52fe5a8c5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:12.216000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:12.216706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.216939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.217180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.224633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:12.226074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:44:12.237144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:12.238109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:44:12.239048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:12.240572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 533/1066] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:44:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:12.243918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:12.259469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:12.351611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf149eb6-f0e9-41b6-99b2-8b8280e406dc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:12.386128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:12.386737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.387015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.387286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.388001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:12.388329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:12.389736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:12.411550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 534/1067] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:44:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:12.412691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:12.513692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba5638-9503-4c21-a9ea-b49b2c00a3e2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:12.536303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:12.547943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b6367-383e-45ff-8a4f-0b386e3aa0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:12.548651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b6367-383e-45ff-8a4f-0b386e3aa0cf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.548934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b6367-383e-45ff-8a4f-0b386e3aa0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.549205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b6367-383e-45ff-8a4f-0b386e3aa0cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.560216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b6367-383e-45ff-8a4f-0b386e3aa0cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:12.561679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b6367-383e-45ff-8a4f-0b386e3aa0cf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ae9b6367-383e-45ff-8a4f-0b386e3aa0cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:12.576281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:12.577241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:44:12.584664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:12.584664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:12.584664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:12.584664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 534/1068] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:12.584955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:12.654586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:12.683872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217ae4-55e1-4d09-b5e0-c3f50bb86b19 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.684229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217ae4-55e1-4d09-b5e0-c3f50bb86b19 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.684392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217ae4-55e1-4d09-b5e0-c3f50bb86b19 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.685731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 535/1069] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:44:12.687555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e611e5-1805-4db3-9a1f-44294862c8fa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:12.690762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dee8e-2d02-4d49-8c32-34bc0803db22 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.691049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dee8e-2d02-4d49-8c32-34bc0803db22 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.691329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dee8e-2d02-4d49-8c32-34bc0803db22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.710272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f21dee8e-2d02-4d49-8c32-34bc0803db22 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:12.711473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f21dee8e-2d02-4d49-8c32-34bc0803db22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:12.724980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:12.725816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.726296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.726767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.727407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:12.727708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:12.729086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:12.745783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 535/1070] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:12.746950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:12.786218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 536/1071] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:12.853656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f52c78-0997-470c-9f83-be425a472e07 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:12.882647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:12.889722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ac5d8-b09e-459d-9c75-1b1be532e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:12.890351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ac5d8-b09e-459d-9c75-1b1be532e861 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:12.890626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ac5d8-b09e-459d-9c75-1b1be532e861 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:12.890895 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ac5d8-b09e-459d-9c75-1b1be532e861 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:12.897682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ac5d8-b09e-459d-9c75-1b1be532e861 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:12.899097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ac5d8-b09e-459d-9c75-1b1be532e861 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b52ac5d8-b09e-459d-9c75-1b1be532e861 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:44:12.910052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:12.910922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:44:12.912392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:12.913376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 536/1072] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:44:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:12.917482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dee8e-2d02-4d49-8c32-34bc0803db22 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.015411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dee8e-2d02-4d49-8c32-34bc0803db22 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.017498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730bb4d-b5b2-443e-846f-ea10546de18e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.017958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730bb4d-b5b2-443e-846f-ea10546de18e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.018304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730bb4d-b5b2-443e-846f-ea10546de18e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.019845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 537/1073] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:44:13.024872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b322b-c9f8-404d-841e-9d6eb7565ca7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.025261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:13.025909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b322b-c9f8-404d-841e-9d6eb7565ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.026419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b322b-c9f8-404d-841e-9d6eb7565ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.046799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-358b322b-c9f8-404d-841e-9d6eb7565ca7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:13.048426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-358b322b-c9f8-404d-841e-9d6eb7565ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:13.054183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.054785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.055056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.055323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.056003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:13.056331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.057752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.069173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 537/1074] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:44:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:13.069957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.117836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 538/1075] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:13.118570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b322b-c9f8-404d-841e-9d6eb7565ca7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.171438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13faa1-e6fe-4ff2-9d49-36b974665b05 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.206701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.207404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.207682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.207951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.219185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:13.223896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:13.225450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:13.226309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 538/1076] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:13.231395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.236112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b322b-c9f8-404d-841e-9d6eb7565ca7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.257394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:13.259159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:13.273057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.273657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.273935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.274210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/95b365b4794c4edebe306e09336c42f3/roles/3c74c79658db45408d0892a3de5fcb85` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.276457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=95b365b4794c4edebe306e09336c42f3, role_id=3c74c79658db45408d0892a3de5fcb85)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.277872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.300647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 539/1077] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:44:13 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/95b365b4794c4edebe306e09336c42f3/roles/3c74c79658db45408d0892a3de5fcb85 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:44:13.358276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3d5b-0d05-489e-8ddc-4b73a95a8857 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.384864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.394565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.395183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.395458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.395729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.396950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:13.397249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.398631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.409120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 539/1078] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:13.415704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.499099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9bda-30e1-4c1e-9e2d-543ee7e577dd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.516423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:13.531179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812880d-ce59-42ea-a059-bdbb3d482ecb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.535442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.536287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.537294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.537740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.568819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.569479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.569727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.570003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] PATH_INFO: `/v3/users/ce282db7581147d092977648f82cea14` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.571189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] RBAC: Authorizing `identity:get_user(user_id=ce282db7581147d092977648f82cea14)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.572708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.574366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 540/1079] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:13 2022] GET /identity/v3/users/ce282db7581147d092977648f82cea14 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:13.579897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.593156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:13.618077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.619644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.682963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e5475-6f36-429d-8581-adbdb0b4f934 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.718936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.719572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.719851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.720143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.721267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbd2b62613b424a90f1185e6cb9b5d7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.722711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.729458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 541/1080] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:13 2022] GET /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:13.729597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 540/1081] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:44:13.729705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.730384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:13.754789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:13.816698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:13.836225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc26dc3-4a09-47e1-87b2-2bf113168bd5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.839066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3675f20-f406-4ad8-bca2-e3a99c6d3d0d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:13.872811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03006687-6e75-41de-9e6b-85b35e1d992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.873292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:13.873479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006687-6e75-41de-9e6b-85b35e1d992f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.873724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006687-6e75-41de-9e6b-85b35e1d992f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.873940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:13.874038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03006687-6e75-41de-9e6b-85b35e1d992f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.874214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:13.874487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7/users/ce282db7581147d092977648f82cea14/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:13.876344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbd2b62613b424a90f1185e6cb9b5d7, user_id=ce282db7581147d092977648f82cea14, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.877752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.885898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03006687-6e75-41de-9e6b-85b35e1d992f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:13.887870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03006687-6e75-41de-9e6b-85b35e1d992f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:13.900291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 541/1082] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:44:13 2022] PUT /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7/users/ce282db7581147d092977648f82cea14/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-03006687-6e75-41de-9e6b-85b35e1d992f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:13.909826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:13.910906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:44:13.911880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:13.911880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:13.911880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:13.916281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 542/1083] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:13.917601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:14.027126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df98fa-b7b0-4e73-b423-e6d0524789b5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:14.063544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e23db5-1839-4f66-9053-0bc60eba6fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:14.064152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e23db5-1839-4f66-9053-0bc60eba6fec None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:14.064470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e23db5-1839-4f66-9053-0bc60eba6fec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:14.064734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e23db5-1839-4f66-9053-0bc60eba6fec None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:14.065380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e23db5-1839-4f66-9053-0bc60eba6fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:14.065683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e23db5-1839-4f66-9053-0bc60eba6fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:14.067107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e23db5-1839-4f66-9053-0bc60eba6fec None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:14.076018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:14.080696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 542/1084] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:15.198168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6231c2-502c-4773-bef3-bfc8436ae483 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:15.198727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6231c2-502c-4773-bef3-bfc8436ae483 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:15.198727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6231c2-502c-4773-bef3-bfc8436ae483 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:15.200046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 543/1085] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:15.205189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fff01-555e-4c09-9b42-1742d15bce40 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:15.205591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fff01-555e-4c09-9b42-1742d15bce40 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:15.205745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fff01-555e-4c09-9b42-1742d15bce40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:15.223996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bb0fff01-555e-4c09-9b42-1742d15bce40 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:15.225207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb0fff01-555e-4c09-9b42-1742d15bce40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:15.289033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 543/1086] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:15.556016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6231c2-502c-4773-bef3-bfc8436ae483 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:15.632456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:15.655019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6231c2-502c-4773-bef3-bfc8436ae483 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:15.687042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24febe4-81e7-4267-998a-32f4a89d86ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:15.687662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24febe4-81e7-4267-998a-32f4a89d86ff None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:15.687934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24febe4-81e7-4267-998a-32f4a89d86ff None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:15.688202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24febe4-81e7-4267-998a-32f4a89d86ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:15.699144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24febe4-81e7-4267-998a-32f4a89d86ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a24febe4-81e7-4267-998a-32f4a89d86ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:15.703896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:15.705081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:15.706057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 544/1087] 10.222.0.50 () {56 vars in 1120 bytes} [Tue Aug 30 01:44:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:15.710939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fff01-555e-4c09-9b42-1742d15bce40 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:15.806343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fff01-555e-4c09-9b42-1742d15bce40 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:15.822607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343474ad-2b9f-4321-8912-cfc737735182 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:15.823111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343474ad-2b9f-4321-8912-cfc737735182 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:15.823231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343474ad-2b9f-4321-8912-cfc737735182 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:15.824752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 545/1088] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:15.829601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af029f0-835e-45d0-86fc-7eed3d91b2c1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:15.829887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af029f0-835e-45d0-86fc-7eed3d91b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:15.830162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af029f0-835e-45d0-86fc-7eed3d91b2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:15.834831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:15.840793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:15.841396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:15.841671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:15.841939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:15.842632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:15.842934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:15.844309 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:15.847981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8af029f0-835e-45d0-86fc-7eed3d91b2c1 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:15.849685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8af029f0-835e-45d0-86fc-7eed3d91b2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:15.852714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 544/1089] 10.222.0.50 () {56 vars in 1115 bytes} [Tue Aug 30 01:44:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:15.858421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:15.914897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 546/1090] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:44:15.953499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153fe80-06f9-4004-8744-1e78d90047f7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:15.985427 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:15.986040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:15.986318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:15.986591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:15.997452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:15.998868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:16.012385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:16.018279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:44:16.019449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:16.019449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:16.019449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:16.019449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 545/1091] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:16.019723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af029f0-835e-45d0-86fc-7eed3d91b2c1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.082622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.117046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af029f0-835e-45d0-86fc-7eed3d91b2c1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.148577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.149162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.149432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.149700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.150327 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:16.150625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.151978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.163028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 547/1092] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:16.168670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.179733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.214022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.267453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8ed90-bf83-414e-8755-4aec220c8ffc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.275681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497518e-605b-423b-b21a-49fcd92747ca None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.303922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.304114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.304574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.304855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.305163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.312683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.313649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.314116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.314274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.314546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.314814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:44:16.324994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:16.325921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 01:44:16.326861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:16.327399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:16.327471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 546/1093] 10.222.0.50 () {56 vars in 1093 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:16.330271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:16.331192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:16.332001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 548/1094] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:16.334550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.337660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.429851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03180762-3fb6-486b-9357-4909941fb746 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.432825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.433913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914387d-b824-486d-abf2-b1d662f885b7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.463394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.463995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.464295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.464568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.465279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:16.465580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.466895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.467001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.467518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.467797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.468069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.468768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:16.469068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.470417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.480139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 549/1095] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:16.483857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 547/1096] 10.222.0.50 () {56 vars in 1133 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:16.485771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.489861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.533157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.584398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba84cc-68eb-47c8-bb9c-00c83a4e1189 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.588618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffb1a8-008e-47cc-a81a-c3068b33e82f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.617233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.617878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.618195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.618420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] PATH_INFO: `/v3/users/ce282db7581147d092977648f82cea14` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.619556 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] RBAC: Authorizing `identity:get_user(user_id=ce282db7581147d092977648f82cea14)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.620990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.621157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.621722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.621998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.622270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.622697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 550/1097] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/users/ce282db7581147d092977648f82cea14 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:16.628235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.652923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.689857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.691280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.725981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb658a-c426-4dea-bb3e-454dd3ef6668 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.742160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.760459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.761059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.761340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.761615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.786463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 548/1098] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:44:16.792470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.828359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.829796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.850675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.888011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bd53e-bc8d-4e34-bf51-6a75627118f2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:16.927014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 551/1099] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:44:16.929005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:16.930076 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:16.930583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:16.931087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:16.933448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:16.941978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:16.943982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:16.950324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:16.964907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:16.966050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:44:16.967168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:16.967168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:44:16.967168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:16.967168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 549/1100] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:16.971543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:17.033915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753344f7-8c05-4341-8e29-44858a78db14 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:17.066537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:17.067156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:17.067431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:17.067747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:17.069108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cf6c7-aa08-4d59-a041-756b7095d760 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:17.069391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] RBAC: Authorizing `identity:get_project(project_id=ecbd2b62613b424a90f1185e6cb9b5d7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:17.070824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:17.072610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 552/1101] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:17.078490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:17.082248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:17.103401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:17.103936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:17.104324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:17.104524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:17.105158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:17.105456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:17.106796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:17.118712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 550/1102] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:17.124198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:17.159982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:17.175768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e590c71-4676-439b-a3f8-17c08986754d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:17.209016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:17.209621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:17.209896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:17.210167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:17.210887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7, user.id=ce282db7581147d092977648f82cea14) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:17.211196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:17.212654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:17.218133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 553/1103] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:17 2022] GET /identity/v3/role_assignments?user.id=ce282db7581147d092977648f82cea14&scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:17.219327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24930b1-c6cf-4fcc-8c29-27d32cc78edb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:17.257379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:17.258027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:17.258304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:17.258576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:17.266184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:17.267726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:44:17.280086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:17.281250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:17.282240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 551/1104] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:44:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:44:17.292277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:17.395338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6e777-a0a0-49be-adc8-2d4ce3528f69 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:17.429540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d2d39-9fb5-4ab9-bb26-1b36841c660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:17.430145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d2d39-9fb5-4ab9-bb26-1b36841c660a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:17.430422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d2d39-9fb5-4ab9-bb26-1b36841c660a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:17.430693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d2d39-9fb5-4ab9-bb26-1b36841c660a None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:17.431379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d2d39-9fb5-4ab9-bb26-1b36841c660a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:17.431681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d2d39-9fb5-4ab9-bb26-1b36841c660a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:17.433138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d2d39-9fb5-4ab9-bb26-1b36841c660a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:17.444732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 554/1105] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:44:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:17.450861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:17.485031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:17.548204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cee60-b70d-4046-8873-8397b9fc3fb2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:17.580270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea52023-e765-402f-ab82-c5e8a70f4c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:17.580879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea52023-e765-402f-ab82-c5e8a70f4c47 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:17.581154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea52023-e765-402f-ab82-c5e8a70f4c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:17.581422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea52023-e765-402f-ab82-c5e8a70f4c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:17.582128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea52023-e765-402f-ab82-c5e8a70f4c47 None admin] RBAC: Adding query filter params (role.id=3c74c79658db45408d0892a3de5fcb85, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=95b365b4794c4edebe306e09336c42f3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:17.582428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea52023-e765-402f-ab82-c5e8a70f4c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:17.583857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea52023-e765-402f-ab82-c5e8a70f4c47 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:17.589505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 552/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:17 2022] GET /identity/v3/role_assignments?user.id=95b365b4794c4edebe306e09336c42f3&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3c74c79658db45408d0892a3de5fcb85 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:19.152628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e22f0-d862-43a6-9468-76e66f764f5f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:19.153158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e22f0-d862-43a6-9468-76e66f764f5f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:19.153158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e22f0-d862-43a6-9468-76e66f764f5f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:19.154493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 555/1107] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:19.159617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37b343-ced9-4d7f-bb22-4b262724ac7c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:19.160014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37b343-ced9-4d7f-bb22-4b262724ac7c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:19.160205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37b343-ced9-4d7f-bb22-4b262724ac7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:19.178874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8b37b343-ced9-4d7f-bb22-4b262724ac7c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:19.180029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8b37b343-ced9-4d7f-bb22-4b262724ac7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:19.243729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 553/1108] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:19.509268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e22f0-d862-43a6-9468-76e66f764f5f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:19.520009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:19.609300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e22f0-d862-43a6-9468-76e66f764f5f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:19.641705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e0218-6d66-454e-9ed6-14751f377615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:19.642324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e0218-6d66-454e-9ed6-14751f377615 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:19.642602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e0218-6d66-454e-9ed6-14751f377615 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:19.642875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e0218-6d66-454e-9ed6-14751f377615 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:19.643972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e0218-6d66-454e-9ed6-14751f377615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:19.645427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e0218-6d66-454e-9ed6-14751f377615 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:19.647631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 556/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:19.653352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37b343-ced9-4d7f-bb22-4b262724ac7c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:19.679228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:19.747728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37b343-ced9-4d7f-bb22-4b262724ac7c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:19.780663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7893f5-fb55-4f32-883e-12a3982fea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:19.781198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7893f5-fb55-4f32-883e-12a3982fea79 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:19.781474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7893f5-fb55-4f32-883e-12a3982fea79 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:19.781745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7893f5-fb55-4f32-883e-12a3982fea79 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:19.782393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7893f5-fb55-4f32-883e-12a3982fea79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:19.783797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7893f5-fb55-4f32-883e-12a3982fea79 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:19.803323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 554/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:44:19 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:21.704628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff07ec-4439-4aa1-8855-5aa5ff88019e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:21.705142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff07ec-4439-4aa1-8855-5aa5ff88019e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:21.705142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff07ec-4439-4aa1-8855-5aa5ff88019e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:21.706416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 557/1111] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:21.711897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da88f72-e00a-45ec-9a52-f0fe130424e6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:21.712329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da88f72-e00a-45ec-9a52-f0fe130424e6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:21.712491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da88f72-e00a-45ec-9a52-f0fe130424e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:21.731035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0da88f72-e00a-45ec-9a52-f0fe130424e6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:21.732213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0da88f72-e00a-45ec-9a52-f0fe130424e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:21.796308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 555/1112] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:22.090254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff07ec-4439-4aa1-8855-5aa5ff88019e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:22.165483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:22.189126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff07ec-4439-4aa1-8855-5aa5ff88019e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:22.221144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:22.221756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:22.222027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:22.222293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] PATH_INFO: `/v3/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:22.223635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] RBAC: Authorizing `identity:get_role(role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:22.225048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:22.226321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 558/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:22 2022] GET /identity/v3/roles/fdf7d3a219d340db99038e400383416b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:22.232051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da88f72-e00a-45ec-9a52-f0fe130424e6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:22.247250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:22.335231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da88f72-e00a-45ec-9a52-f0fe130424e6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:22.368151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:22.368729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:22.369244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:22.369517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] PATH_INFO: `/v3/users/23f551f0dc94461abc071b83a909c6be` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:22.383903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] RBAC: Authorizing `identity:get_user(user_id=23f551f0dc94461abc071b83a909c6be)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:22.385347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:22.387735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 556/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:22 2022] GET /identity/v3/users/23f551f0dc94461abc071b83a909c6be => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:22.393546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:22.488871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468696c0-4a85-4752-a892-189a7f60f883 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:22.521192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:22.521831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:22.522109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:22.522382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:22.536025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:22.588140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:22.589607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:22.683815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 559/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:44:22.689675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:22.756662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:22.785233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d56f5-a75c-4ffe-b1fc-eddfd3af877a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:22.822097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198459c9-4f47-4069-8ff1-4514bacec302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:22.822711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198459c9-4f47-4069-8ff1-4514bacec302 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:22.822983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198459c9-4f47-4069-8ff1-4514bacec302 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:22.823251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198459c9-4f47-4069-8ff1-4514bacec302 None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:22.824359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198459c9-4f47-4069-8ff1-4514bacec302 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbd2b62613b424a90f1185e6cb9b5d7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:22.825771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198459c9-4f47-4069-8ff1-4514bacec302 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:22.827365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 557/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:22 2022] GET /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:22.834064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:22.877524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:22.930638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a8f55-5974-4ab6-b7fa-48ee63ba6331 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:22.963805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2624d3-448e-420a-9045-c3a68cc6a48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:22.964361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2624d3-448e-420a-9045-c3a68cc6a48b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:22.964643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2624d3-448e-420a-9045-c3a68cc6a48b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:22.964919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2624d3-448e-420a-9045-c3a68cc6a48b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:22.965631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2624d3-448e-420a-9045-c3a68cc6a48b None admin] RBAC: Adding query filter params (role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7, user.id=23f551f0dc94461abc071b83a909c6be) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:22.965939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2624d3-448e-420a-9045-c3a68cc6a48b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:22.967368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2624d3-448e-420a-9045-c3a68cc6a48b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:22.972672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 560/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:22 2022] GET /identity/v3/role_assignments?user.id=23f551f0dc94461abc071b83a909c6be&scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7&role.id=fdf7d3a219d340db99038e400383416b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:24.821029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffcaf7-c026-4910-bc6b-a035be1c5199 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:24.821599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffcaf7-c026-4910-bc6b-a035be1c5199 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:24.821599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffcaf7-c026-4910-bc6b-a035be1c5199 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:24.822886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 558/1118] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:24.827987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0068e03-7204-4669-ad8a-b919fabe7b40 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:24.828430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0068e03-7204-4669-ad8a-b919fabe7b40 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:24.828570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0068e03-7204-4669-ad8a-b919fabe7b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:24.846261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a0068e03-7204-4669-ad8a-b919fabe7b40 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:24.847432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a0068e03-7204-4669-ad8a-b919fabe7b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:24.910647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 561/1119] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:25.173562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffcaf7-c026-4910-bc6b-a035be1c5199 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:25.269472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffcaf7-c026-4910-bc6b-a035be1c5199 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:25.302238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:25.302851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:25.303126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:25.303395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] PATH_INFO: `/v3/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:25.304785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] RBAC: Authorizing `identity:get_role(role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:25.306123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:25.307419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 559/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:25 2022] GET /identity/v3/roles/fdf7d3a219d340db99038e400383416b => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:25.313055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0068e03-7204-4669-ad8a-b919fabe7b40 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:25.351017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:25.407472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0068e03-7204-4669-ad8a-b919fabe7b40 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:25.440253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:25.440823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:25.441095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:25.441361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] PATH_INFO: `/v3/users/23f551f0dc94461abc071b83a909c6be` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:25.442447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] RBAC: Authorizing `identity:get_user(user_id=23f551f0dc94461abc071b83a909c6be)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:25.443898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:25.445458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 562/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:25 2022] GET /identity/v3/users/23f551f0dc94461abc071b83a909c6be => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:25.451102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:25.462284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:25.547874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0a1b7-33c1-43d0-abb7-ed723fa00f48 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:25.580119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d318d2-cc6d-4490-9f69-81fa0abbebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:25.580760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d318d2-cc6d-4490-9f69-81fa0abbebd1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:25.581039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d318d2-cc6d-4490-9f69-81fa0abbebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:25.581313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d318d2-cc6d-4490-9f69-81fa0abbebd1 None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:25.582405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d318d2-cc6d-4490-9f69-81fa0abbebd1 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbd2b62613b424a90f1185e6cb9b5d7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:25.584409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d318d2-cc6d-4490-9f69-81fa0abbebd1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:25.586007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 560/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:25 2022] GET /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:25.591796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:25.688254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d23805-1d4f-4918-bca7-4a974d00975c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:25.698577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:25.733824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d808502-e156-485b-9feb-e0f64e921bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:25.734767 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808502-e156-485b-9feb-e0f64e921bb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:25.735178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808502-e156-485b-9feb-e0f64e921bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:25.735601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808502-e156-485b-9feb-e0f64e921bb2 None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7/users/23f551f0dc94461abc071b83a909c6be/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:25.738306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d808502-e156-485b-9feb-e0f64e921bb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbd2b62613b424a90f1185e6cb9b5d7, user_id=23f551f0dc94461abc071b83a909c6be, role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:25.739787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d808502-e156-485b-9feb-e0f64e921bb2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:25.754228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 563/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:44:25 2022] PUT /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7/users/23f551f0dc94461abc071b83a909c6be/roles/fdf7d3a219d340db99038e400383416b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:44:27.603591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2358-87d7-478c-85b7-5552d4e4bed6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:27.604118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2358-87d7-478c-85b7-5552d4e4bed6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:27.604118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2358-87d7-478c-85b7-5552d4e4bed6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:27.605457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 561/1124] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:27.610604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a694e-4aa6-4d4e-98bd-22a82d6ca447 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:27.610967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a694e-4aa6-4d4e-98bd-22a82d6ca447 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:27.611151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a694e-4aa6-4d4e-98bd-22a82d6ca447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:27.629055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-992a694e-4aa6-4d4e-98bd-22a82d6ca447 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:27.630240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-992a694e-4aa6-4d4e-98bd-22a82d6ca447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:27.693594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 564/1125] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:27.960189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf2358-87d7-478c-85b7-5552d4e4bed6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:28.034850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:28.057979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf2358-87d7-478c-85b7-5552d4e4bed6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:28.090995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:28.091610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:28.091888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:28.092162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] PATH_INFO: `/v3/roles/fdf7d3a219d340db99038e400383416b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:28.093634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] RBAC: Authorizing `identity:get_role(role_id=fdf7d3a219d340db99038e400383416b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:28.094979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:28.096294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 562/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:27 2022] GET /identity/v3/roles/fdf7d3a219d340db99038e400383416b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:28.102030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a694e-4aa6-4d4e-98bd-22a82d6ca447 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:28.192056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:28.199017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a694e-4aa6-4d4e-98bd-22a82d6ca447 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:28.231709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:28.232364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:28.232647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:28.232924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] PATH_INFO: `/v3/users/23f551f0dc94461abc071b83a909c6be` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:28.234035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] RBAC: Authorizing `identity:get_user(user_id=23f551f0dc94461abc071b83a909c6be)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:28.235469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:28.237320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 565/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:28 2022] GET /identity/v3/users/23f551f0dc94461abc071b83a909c6be => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:28.243446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:28.339104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8dcfe-bdcc-4dcd-bdf2-f58b31f61c25 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:28.371716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:28.372332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:28.372612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:28.372884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:28.386809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:28.437448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:28.438876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:28.532219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 563/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:44:28.538530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:28.635473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b4ad0-746e-4085-9399-3ec3e964ebae None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:28.667982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a6b6e-610b-4417-8d90-af3e17f666bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:28.668643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a6b6e-610b-4417-8d90-af3e17f666bd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:28.668922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a6b6e-610b-4417-8d90-af3e17f666bd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:28.669196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a6b6e-610b-4417-8d90-af3e17f666bd None admin] PATH_INFO: `/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:28.670298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a6b6e-610b-4417-8d90-af3e17f666bd None admin] RBAC: Authorizing `identity:get_project(project_id=ecbd2b62613b424a90f1185e6cb9b5d7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:28.671723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a6b6e-610b-4417-8d90-af3e17f666bd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:28.673363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 566/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:28 2022] GET /identity/v3/projects/ecbd2b62613b424a90f1185e6cb9b5d7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:28.679959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:28.723177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:28.775254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b2fa-6f19-48a5-9b9f-de8787e07815 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:28.807981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51f9d6-26ae-4b77-ac5c-630cc9f3d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:28.808541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f9d6-26ae-4b77-ac5c-630cc9f3d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:28.808821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f9d6-26ae-4b77-ac5c-630cc9f3d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:28.809092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51f9d6-26ae-4b77-ac5c-630cc9f3d4a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:28.809802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f9d6-26ae-4b77-ac5c-630cc9f3d4a8 None admin] RBAC: Adding query filter params (role.id=fdf7d3a219d340db99038e400383416b, scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7, user.id=23f551f0dc94461abc071b83a909c6be) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:28.810106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f9d6-26ae-4b77-ac5c-630cc9f3d4a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:28.811516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51f9d6-26ae-4b77-ac5c-630cc9f3d4a8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:28.817014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 564/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:28 2022] GET /identity/v3/role_assignments?user.id=23f551f0dc94461abc071b83a909c6be&scope.project.id=ecbd2b62613b424a90f1185e6cb9b5d7&role.id=fdf7d3a219d340db99038e400383416b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:30.727515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e0336-95d6-46b0-96ea-766621411a6a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:30.728025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e0336-95d6-46b0-96ea-766621411a6a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:30.728025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e0336-95d6-46b0-96ea-766621411a6a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:30.729328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 567/1131] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:30.734517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be5e37-5efe-4a95-bfaf-e5e90b66ce5f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:30.734941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be5e37-5efe-4a95-bfaf-e5e90b66ce5f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:30.735036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be5e37-5efe-4a95-bfaf-e5e90b66ce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:30.753376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-16be5e37-5efe-4a95-bfaf-e5e90b66ce5f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:30.754533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-16be5e37-5efe-4a95-bfaf-e5e90b66ce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:30.819911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 565/1132] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:31.083989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e0336-95d6-46b0-96ea-766621411a6a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:31.116454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:31.183663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e0336-95d6-46b0-96ea-766621411a6a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:31.215585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab62fb6-632d-4934-a4eb-a13272e216a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:31.216194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62fb6-632d-4934-a4eb-a13272e216a2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:31.216499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62fb6-632d-4934-a4eb-a13272e216a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:31.216769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62fb6-632d-4934-a4eb-a13272e216a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:31.217850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab62fb6-632d-4934-a4eb-a13272e216a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:31.219268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab62fb6-632d-4934-a4eb-a13272e216a2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:31.220828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 568/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:31.227217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be5e37-5efe-4a95-bfaf-e5e90b66ce5f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:31.264594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:31.322329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be5e37-5efe-4a95-bfaf-e5e90b66ce5f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:31.354254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd21149-e483-4b31-bd73-0dc431d20239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:31.354794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd21149-e483-4b31-bd73-0dc431d20239 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:31.355072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd21149-e483-4b31-bd73-0dc431d20239 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:31.355343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd21149-e483-4b31-bd73-0dc431d20239 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:31.355982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd21149-e483-4b31-bd73-0dc431d20239 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:31.357417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd21149-e483-4b31-bd73-0dc431d20239 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:31.383613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 566/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:44:31 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:33.275560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570c44-c07f-4913-8988-459a255bf611 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:33.276084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570c44-c07f-4913-8988-459a255bf611 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:33.276084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570c44-c07f-4913-8988-459a255bf611 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:33.277539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 569/1135] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:33.282636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d20f17-2d89-42a2-9e76-9b8e57e3ba73 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:33.283071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d20f17-2d89-42a2-9e76-9b8e57e3ba73 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:33.283172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d20f17-2d89-42a2-9e76-9b8e57e3ba73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:33.301095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d6d20f17-2d89-42a2-9e76-9b8e57e3ba73 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:33.302305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6d20f17-2d89-42a2-9e76-9b8e57e3ba73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:33.366797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 567/1136] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:33.652808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570c44-c07f-4913-8988-459a255bf611 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:33.749590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:33.753634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570c44-c07f-4913-8988-459a255bf611 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:33.787825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07044d-c01c-4b23-b18f-ec6f6193e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:33.788481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07044d-c01c-4b23-b18f-ec6f6193e7ce None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:33.788779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07044d-c01c-4b23-b18f-ec6f6193e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:33.789067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07044d-c01c-4b23-b18f-ec6f6193e7ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:33.790169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07044d-c01c-4b23-b18f-ec6f6193e7ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:33.791589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07044d-c01c-4b23-b18f-ec6f6193e7ce None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:33.793168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 570/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:44:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:33.798812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d20f17-2d89-42a2-9e76-9b8e57e3ba73 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:33.819590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:33.895606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d20f17-2d89-42a2-9e76-9b8e57e3ba73 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:33.928696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef60b17-5985-428d-acc2-5184f0d4137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:33.929229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef60b17-5985-428d-acc2-5184f0d4137c None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:33.929506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef60b17-5985-428d-acc2-5184f0d4137c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:33.929779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef60b17-5985-428d-acc2-5184f0d4137c None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:33.930433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef60b17-5985-428d-acc2-5184f0d4137c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:33.931843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef60b17-5985-428d-acc2-5184f0d4137c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:33.951861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 568/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:44:33 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:35.876067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adb9c2-5991-4de4-b22c-a4cafc1964cb None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:35.876648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adb9c2-5991-4de4-b22c-a4cafc1964cb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:35.876747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adb9c2-5991-4de4-b22c-a4cafc1964cb None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:35.878061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 571/1139] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:35.883678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d920cd-2e59-46d8-92ad-639928880e6c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:35.884108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d920cd-2e59-46d8-92ad-639928880e6c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:35.884223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d920cd-2e59-46d8-92ad-639928880e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:35.901946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f3d920cd-2e59-46d8-92ad-639928880e6c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:35.903110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f3d920cd-2e59-46d8-92ad-639928880e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:35.966710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 569/1140] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:36.233081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adb9c2-5991-4de4-b22c-a4cafc1964cb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:36.330490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adb9c2-5991-4de4-b22c-a4cafc1964cb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:36.362821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:36.363433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:36.363705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:36.363976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:36.374161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:36.378861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:36.380020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:36.381204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 572/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:36.386071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d920cd-2e59-46d8-92ad-639928880e6c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:36.393963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:36.481519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d920cd-2e59-46d8-92ad-639928880e6c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:36.513268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:36.513796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:36.514069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:36.514336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:36.514981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:36.515279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:36.516646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:36.524906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 570/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:36.530492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:36.558384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:36.628008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6134177-597d-4d7a-8094-68a0b26060fa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:36.660565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:36.661769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:36.662040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:36.662308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] PATH_INFO: `/v3/users/f56609e59d584b5db32d4c61a9bef659` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:36.674563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] RBAC: Authorizing `identity:get_user(user_id=f56609e59d584b5db32d4c61a9bef659)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:36.675992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:36.677580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 573/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:36 2022] GET /identity/v3/users/f56609e59d584b5db32d4c61a9bef659 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:36.683445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:36.747230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:36.779643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f186c-7d10-46be-9931-82cc869be712 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:36.811352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:36.811945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:36.812222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:36.812526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:36.876731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:36.878144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:36.938834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:36.972361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 571/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:44:36.978298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:37.059146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:37.074167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa535aff-d491-43ed-aeb0-39336d0f32b1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:37.107217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e02889-d6e3-4c85-adbc-0b7eff5675cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:37.107829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e02889-d6e3-4c85-adbc-0b7eff5675cf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:37.108102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e02889-d6e3-4c85-adbc-0b7eff5675cf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:37.108394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e02889-d6e3-4c85-adbc-0b7eff5675cf None admin] PATH_INFO: `/v3/projects/fd6a0da7f65d4390966336168d882522` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:37.109484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e02889-d6e3-4c85-adbc-0b7eff5675cf None admin] RBAC: Authorizing `identity:get_project(project_id=fd6a0da7f65d4390966336168d882522)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:37.110907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e02889-d6e3-4c85-adbc-0b7eff5675cf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:37.112782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 574/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:36 2022] GET /identity/v3/projects/fd6a0da7f65d4390966336168d882522 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:37.118971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:37.214107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13613-b089-4331-8fce-1e18962e8c5c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:37.246584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7f666-c00d-47ca-baee-4d5cedc9dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:37.247166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f666-c00d-47ca-baee-4d5cedc9dc17 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:37.247442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f666-c00d-47ca-baee-4d5cedc9dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:37.247711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f666-c00d-47ca-baee-4d5cedc9dc17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:37.248447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7f666-c00d-47ca-baee-4d5cedc9dc17 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=fd6a0da7f65d4390966336168d882522, user.id=f56609e59d584b5db32d4c61a9bef659) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:37.248754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7f666-c00d-47ca-baee-4d5cedc9dc17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:37.250144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7f666-c00d-47ca-baee-4d5cedc9dc17 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:37.255814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 572/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:37 2022] GET /identity/v3/role_assignments?user.id=f56609e59d584b5db32d4c61a9bef659&scope.project.id=fd6a0da7f65d4390966336168d882522&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:39.120778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e8fd-d30e-49c2-b6a3-0a89ba9ecaa6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:39.121361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e8fd-d30e-49c2-b6a3-0a89ba9ecaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:39.121361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e8fd-d30e-49c2-b6a3-0a89ba9ecaa6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:39.122624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 575/1147] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:39.127741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6494a-5b02-4500-902a-6e2989521b99 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:39.128147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6494a-5b02-4500-902a-6e2989521b99 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:39.128318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6494a-5b02-4500-902a-6e2989521b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:39.147052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-45b6494a-5b02-4500-902a-6e2989521b99 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:39.148212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-45b6494a-5b02-4500-902a-6e2989521b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:39.172403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:39.214203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 573/1148] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:39.477121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7e8fd-d30e-49c2-b6a3-0a89ba9ecaa6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:39.574237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7e8fd-d30e-49c2-b6a3-0a89ba9ecaa6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:39.605515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:39.606127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:39.606481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:39.606756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:39.617020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:39.621698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:39.622781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:39.623964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 576/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:39.628561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6494a-5b02-4500-902a-6e2989521b99 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:39.721565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6494a-5b02-4500-902a-6e2989521b99 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:39.755537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:39.756122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:39.756428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:39.756694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:39.757340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:39.757633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:39.758965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:39.766845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 574/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:39.772405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:39.788451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:39.868981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704c73a-494f-4512-b396-4ab2be042c17 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:39.900783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:39.901386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:39.901658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:39.901923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] PATH_INFO: `/v3/users/f56609e59d584b5db32d4c61a9bef659` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:39.903007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] RBAC: Authorizing `identity:get_user(user_id=f56609e59d584b5db32d4c61a9bef659)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:39.904455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:39.905974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 577/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:39 2022] GET /identity/v3/users/f56609e59d584b5db32d4c61a9bef659 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:39.912055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:39.937721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:40.009278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cee68f-3380-438b-90a8-7287d180ab45 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:40.041356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d37f8-2e65-4e3b-81a9-ef86cf8698c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:40.041962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d37f8-2e65-4e3b-81a9-ef86cf8698c4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:40.042237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d37f8-2e65-4e3b-81a9-ef86cf8698c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:40.042506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d37f8-2e65-4e3b-81a9-ef86cf8698c4 None admin] PATH_INFO: `/v3/projects/fd6a0da7f65d4390966336168d882522` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:40.043589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d37f8-2e65-4e3b-81a9-ef86cf8698c4 None admin] RBAC: Authorizing `identity:get_project(project_id=fd6a0da7f65d4390966336168d882522)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:40.045003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d37f8-2e65-4e3b-81a9-ef86cf8698c4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:40.046583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 575/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:39 2022] GET /identity/v3/projects/fd6a0da7f65d4390966336168d882522 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:40.052919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:40.130487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:40.148707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc68e99-b822-47be-8465-60fff4eae24a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:40.181662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25466a81-3c9c-4a1f-8c11-216d896a48df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:40.182297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25466a81-3c9c-4a1f-8c11-216d896a48df None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:40.182573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25466a81-3c9c-4a1f-8c11-216d896a48df None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:40.182844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25466a81-3c9c-4a1f-8c11-216d896a48df None admin] PATH_INFO: `/v3/projects/fd6a0da7f65d4390966336168d882522/users/f56609e59d584b5db32d4c61a9bef659/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:40.184727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25466a81-3c9c-4a1f-8c11-216d896a48df None admin] RBAC: Authorizing `identity:create_grant(project_id=fd6a0da7f65d4390966336168d882522, user_id=f56609e59d584b5db32d4c61a9bef659, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:40.186141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25466a81-3c9c-4a1f-8c11-216d896a48df None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:40.198984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 578/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:44:40 2022] PUT /identity/v3/projects/fd6a0da7f65d4390966336168d882522/users/f56609e59d584b5db32d4c61a9bef659/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:44:42.056849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014e3f-cca5-4463-9400-2d0399d1cf39 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:42.057377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014e3f-cca5-4463-9400-2d0399d1cf39 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:42.057377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014e3f-cca5-4463-9400-2d0399d1cf39 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:42.058719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 576/1154] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:42.063908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970c181-f5df-49e9-9a50-18650071343d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:42.064326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970c181-f5df-49e9-9a50-18650071343d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:42.064480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970c181-f5df-49e9-9a50-18650071343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:42.082440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0970c181-f5df-49e9-9a50-18650071343d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:42.084169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0970c181-f5df-49e9-9a50-18650071343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:42.147396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 579/1155] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:42.413058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc014e3f-cca5-4463-9400-2d0399d1cf39 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:42.502864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:42.512533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc014e3f-cca5-4463-9400-2d0399d1cf39 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:42.545074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:42.545681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:42.545955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:42.546224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:42.556686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:42.561347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:42.562412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:42.563609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 577/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:42.568845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970c181-f5df-49e9-9a50-18650071343d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:42.634297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:42.662544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970c181-f5df-49e9-9a50-18650071343d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:42.695180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:42.695714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:42.695988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:42.696280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:42.696927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:42.697248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:42.698601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:42.706185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 580/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:44:42.711972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:42.808114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204ead-3862-4a39-9d0d-225d20ec326e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:42.841191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:42.841812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:42.842093 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:42.842370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] PATH_INFO: `/v3/users/f56609e59d584b5db32d4c61a9bef659` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:42.843543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] RBAC: Authorizing `identity:get_user(user_id=f56609e59d584b5db32d4c61a9bef659)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:42.845004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:42.846582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 578/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:42 2022] GET /identity/v3/users/f56609e59d584b5db32d4c61a9bef659 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:42.853122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:42.949071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6874b8-9c5c-47d8-a2d6-ad6303569390 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:42.981164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:42.981747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:42.982023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:42.982291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:42.984758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:43.047554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:43.049035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:43.150258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 581/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:44:43.156217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:43.178722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:43.252509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c87ef-55cc-4002-ad07-1806709398cd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:43.283915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e801428-4318-4b43-8386-1847f2b3b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:43.284555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e801428-4318-4b43-8386-1847f2b3b6ab None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:43.284832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e801428-4318-4b43-8386-1847f2b3b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:43.285102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e801428-4318-4b43-8386-1847f2b3b6ab None admin] PATH_INFO: `/v3/projects/fd6a0da7f65d4390966336168d882522` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:43.286772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e801428-4318-4b43-8386-1847f2b3b6ab None admin] RBAC: Authorizing `identity:get_project(project_id=fd6a0da7f65d4390966336168d882522)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:43.288169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e801428-4318-4b43-8386-1847f2b3b6ab None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:43.289804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 579/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:43 2022] GET /identity/v3/projects/fd6a0da7f65d4390966336168d882522 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:43.295868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:43.327721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:43.391184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3df3b-f044-4605-9689-8f921e196d28 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:43.423678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413286b-d611-4da0-9073-9cc94cd8f09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:43.424285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413286b-d611-4da0-9073-9cc94cd8f09c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:43.424558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413286b-d611-4da0-9073-9cc94cd8f09c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:43.424826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413286b-d611-4da0-9073-9cc94cd8f09c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:43.425534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413286b-d611-4da0-9073-9cc94cd8f09c None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=fd6a0da7f65d4390966336168d882522, user.id=f56609e59d584b5db32d4c61a9bef659) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:43.425835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413286b-d611-4da0-9073-9cc94cd8f09c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:43.427268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413286b-d611-4da0-9073-9cc94cd8f09c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:43.432751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 582/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:43 2022] GET /identity/v3/role_assignments?user.id=f56609e59d584b5db32d4c61a9bef659&scope.project.id=fd6a0da7f65d4390966336168d882522&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:45.329558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68c3d-b803-47d3-8dff-89334a109416 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:45.329967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68c3d-b803-47d3-8dff-89334a109416 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:45.330137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68c3d-b803-47d3-8dff-89334a109416 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:45.331504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 580/1162] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:45.337169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f8d17-c4b2-4b8f-83ce-e530cddde2bf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:45.337575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f8d17-c4b2-4b8f-83ce-e530cddde2bf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:45.337722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f8d17-c4b2-4b8f-83ce-e530cddde2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:45.355880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-003f8d17-c4b2-4b8f-83ce-e530cddde2bf None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:45.357089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-003f8d17-c4b2-4b8f-83ce-e530cddde2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:45.421035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 583/1163] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:45.685519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68c3d-b803-47d3-8dff-89334a109416 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:45.770545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:45.783027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68c3d-b803-47d3-8dff-89334a109416 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:45.815221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:45.815807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:45.816080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:45.816379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:45.880372 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:45.881785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:45.962307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:45.974994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 581/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:44:45.980884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f8d17-c4b2-4b8f-83ce-e530cddde2bf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:46.007914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:46.076394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f8d17-c4b2-4b8f-83ce-e530cddde2bf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:46.108537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a199d7-9ab4-4ec4-90a6-d54e4d44f04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:46.109143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a199d7-9ab4-4ec4-90a6-d54e4d44f04c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:46.109415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a199d7-9ab4-4ec4-90a6-d54e4d44f04c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:46.109683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a199d7-9ab4-4ec4-90a6-d54e4d44f04c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:46.116881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a199d7-9ab4-4ec4-90a6-d54e4d44f04c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:46.118309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a199d7-9ab4-4ec4-90a6-d54e4d44f04c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-04a199d7-9ab4-4ec4-90a6-d54e4d44f04c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:46.129194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:46.130377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:44:46.131482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 584/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:44:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:46.136022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:46.230328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c473b44-c4fd-4208-9564-7c48d380e68c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:46.262717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826af5c2-b39c-40fa-8392-0582ae6c0849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:46.263305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af5c2-b39c-40fa-8392-0582ae6c0849 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:46.263578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af5c2-b39c-40fa-8392-0582ae6c0849 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:46.263843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af5c2-b39c-40fa-8392-0582ae6c0849 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:46.264497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826af5c2-b39c-40fa-8392-0582ae6c0849 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:46.264794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826af5c2-b39c-40fa-8392-0582ae6c0849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:46.266130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826af5c2-b39c-40fa-8392-0582ae6c0849 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:46.273373 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 582/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Tue Aug 30 01:44:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:48.128235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de18de-cdc5-4f9a-8d3d-dd230dae60c6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:48.128779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de18de-cdc5-4f9a-8d3d-dd230dae60c6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:48.128779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de18de-cdc5-4f9a-8d3d-dd230dae60c6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:48.130618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 585/1167] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:48.135885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae870e-22e6-4a32-8ac9-27d8a2defb04 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:48.136290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae870e-22e6-4a32-8ac9-27d8a2defb04 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:48.136468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae870e-22e6-4a32-8ac9-27d8a2defb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:48.154628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b8ae870e-22e6-4a32-8ac9-27d8a2defb04 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:48.155784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b8ae870e-22e6-4a32-8ac9-27d8a2defb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:48.199913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:48.219312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 583/1168] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:48.488286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de18de-cdc5-4f9a-8d3d-dd230dae60c6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:48.571934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:48.584954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de18de-cdc5-4f9a-8d3d-dd230dae60c6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:48.617398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f33d-2dd1-445b-a726-b3a1e265208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:48.617987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f33d-2dd1-445b-a726-b3a1e265208b None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:48.618263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f33d-2dd1-445b-a726-b3a1e265208b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:48.618536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f33d-2dd1-445b-a726-b3a1e265208b None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:48.619203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f33d-2dd1-445b-a726-b3a1e265208b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:48.620581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f33d-2dd1-445b-a726-b3a1e265208b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:48.635558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 586/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Tue Aug 30 01:44:48 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:50.527588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062c17d-cfa1-45b4-ba1e-38da7ffba3eb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:50.528113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062c17d-cfa1-45b4-ba1e-38da7ffba3eb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:50.528113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062c17d-cfa1-45b4-ba1e-38da7ffba3eb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:50.529488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 584/1170] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:50.534561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c183b-699f-4908-b57e-e38ff6f3d9cb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:50.534971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c183b-699f-4908-b57e-e38ff6f3d9cb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:50.535105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c183b-699f-4908-b57e-e38ff6f3d9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:50.552983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a09c183b-699f-4908-b57e-e38ff6f3d9cb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:50.554078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a09c183b-699f-4908-b57e-e38ff6f3d9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:50.617939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 587/1171] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:50.885310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062c17d-cfa1-45b4-ba1e-38da7ffba3eb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:50.981098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062c17d-cfa1-45b4-ba1e-38da7ffba3eb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:51.013967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc13822-1ec1-4056-923e-f60b6fe9775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:51.014583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc13822-1ec1-4056-923e-f60b6fe9775e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:51.014859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc13822-1ec1-4056-923e-f60b6fe9775e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:51.015131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc13822-1ec1-4056-923e-f60b6fe9775e None admin] PATH_INFO: `/v3/domains/8b611fd474684f538118c8c3044f1197` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:51.025110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc13822-1ec1-4056-923e-f60b6fe9775e None admin] RBAC: Authorizing `identity:get_domain(domain_id=8b611fd474684f538118c8c3044f1197)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:51.026535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc13822-1ec1-4056-923e-f60b6fe9775e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:51.028089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 585/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Tue Aug 30 01:44:50 2022] GET /identity/v3/domains/8b611fd474684f538118c8c3044f1197 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:51.033649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c183b-699f-4908-b57e-e38ff6f3d9cb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:51.108825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:51.131068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c183b-699f-4908-b57e-e38ff6f3d9cb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:51.164671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53861d2d-95ad-4997-a4eb-9f75adc79f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:51.165208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861d2d-95ad-4997-a4eb-9f75adc79f3a None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:51.165491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861d2d-95ad-4997-a4eb-9f75adc79f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:51.165764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53861d2d-95ad-4997-a4eb-9f75adc79f3a None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:51.166420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53861d2d-95ad-4997-a4eb-9f75adc79f3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:51.167848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53861d2d-95ad-4997-a4eb-9f75adc79f3a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:51.193822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 588/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:44:51 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:53.091162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21b3f1-fd63-438d-85ab-cbc0dd3c2956 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:53.091718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21b3f1-fd63-438d-85ab-cbc0dd3c2956 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:53.091718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21b3f1-fd63-438d-85ab-cbc0dd3c2956 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:53.093062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 586/1174] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:53.098795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f28ac4-0891-4176-ab58-ecf0d346fef5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:53.099175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f28ac4-0891-4176-ab58-ecf0d346fef5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:53.099350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f28ac4-0891-4176-ab58-ecf0d346fef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:53.117250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a2f28ac4-0891-4176-ab58-ecf0d346fef5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:53.118408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a2f28ac4-0891-4176-ab58-ecf0d346fef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:53.182022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 589/1175] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:53.450815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21b3f1-fd63-438d-85ab-cbc0dd3c2956 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:53.494091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:53.550156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21b3f1-fd63-438d-85ab-cbc0dd3c2956 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:53.584843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de1e33-eaea-4654-bb8c-d42ca415e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:53.585474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1e33-eaea-4654-bb8c-d42ca415e116 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:53.585758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1e33-eaea-4654-bb8c-d42ca415e116 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:53.586036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de1e33-eaea-4654-bb8c-d42ca415e116 None admin] PATH_INFO: `/v3/domains/8b611fd474684f538118c8c3044f1197` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:53.587172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de1e33-eaea-4654-bb8c-d42ca415e116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8b611fd474684f538118c8c3044f1197)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:53.588659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de1e33-eaea-4654-bb8c-d42ca415e116 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:53.590292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 587/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Tue Aug 30 01:44:53 2022] GET /identity/v3/domains/8b611fd474684f538118c8c3044f1197 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:53.597055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f28ac4-0891-4176-ab58-ecf0d346fef5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:53.653113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:53.693038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f28ac4-0891-4176-ab58-ecf0d346fef5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:53.724460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943e3aa-9eb1-459a-a62d-cb54e1490afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:53.724997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e3aa-9eb1-459a-a62d-cb54e1490afa None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:53.725269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e3aa-9eb1-459a-a62d-cb54e1490afa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:53.725534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e3aa-9eb1-459a-a62d-cb54e1490afa None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:53.726183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943e3aa-9eb1-459a-a62d-cb54e1490afa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:53.727592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943e3aa-9eb1-459a-a62d-cb54e1490afa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:53.749032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 590/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:44:53 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:44:55.653660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d1ad-b61b-44b0-8209-640b9789180b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:55.654214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d1ad-b61b-44b0-8209-640b9789180b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:55.654214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543d1ad-b61b-44b0-8209-640b9789180b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:55.655455 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 588/1178] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:55.660534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246f3d-7d0d-4548-a196-809f7c7a7d10 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:55.661471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246f3d-7d0d-4548-a196-809f7c7a7d10 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:55.661668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246f3d-7d0d-4548-a196-809f7c7a7d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:55.680004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f9246f3d-7d0d-4548-a196-809f7c7a7d10 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:55.681228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f9246f3d-7d0d-4548-a196-809f7c7a7d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:55.745848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 591/1179] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:56.009819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d1ad-b61b-44b0-8209-640b9789180b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:56.107994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543d1ad-b61b-44b0-8209-640b9789180b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:56.118021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:56.140393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:56.141002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:56.141280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:56.141549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:56.152079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:56.156785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:56.157908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:56.159069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 589/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:56.164332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246f3d-7d0d-4548-a196-809f7c7a7d10 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:56.208021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:56.258630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246f3d-7d0d-4548-a196-809f7c7a7d10 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:56.291838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:56.292409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:56.292685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:56.292957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:56.293609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:56.293904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:56.295261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:56.303189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 592/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:56.309015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:56.408225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79393a1-f413-400e-83aa-e438e360f589 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:56.439877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:56.440553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:56.440836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:56.441109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] PATH_INFO: `/v3/users/02b70e4c6ee24156b986b028d366dcbc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:56.454217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] RBAC: Authorizing `identity:get_user(user_id=02b70e4c6ee24156b986b028d366dcbc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:56.455634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:56.457234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 590/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:56 2022] GET /identity/v3/users/02b70e4c6ee24156b986b028d366dcbc => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:56.463618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:56.557694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffb0a-7a8d-445e-8163-181426e3ccc8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:56.565942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:56.590518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:56.591115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:56.591389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:56.591658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:56.655770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:56.657216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:56.750420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 593/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:44:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:44:56.756896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:56.783575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:56.852193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b87f-2233-4b00-bf31-5545671f7415 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:56.885245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55343368-3c6d-4ae8-af60-b8121e115b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:56.885857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55343368-3c6d-4ae8-af60-b8121e115b7a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:56.886134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55343368-3c6d-4ae8-af60-b8121e115b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:56.886405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55343368-3c6d-4ae8-af60-b8121e115b7a None admin] PATH_INFO: `/v3/projects/1eae69c100414f56ad94e32e5e70143f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:56.887498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55343368-3c6d-4ae8-af60-b8121e115b7a None admin] RBAC: Authorizing `identity:get_project(project_id=1eae69c100414f56ad94e32e5e70143f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:56.888937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55343368-3c6d-4ae8-af60-b8121e115b7a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:56.890530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 591/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:56 2022] GET /identity/v3/projects/1eae69c100414f56ad94e32e5e70143f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:56.896566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:56.906578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:56.991142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a515-f403-4998-aed2-59a9ff6b99b9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:57.027958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d54f8-5eaa-4452-9fd2-a6b5fdd0b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:57.028595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d54f8-5eaa-4452-9fd2-a6b5fdd0b816 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:57.028871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d54f8-5eaa-4452-9fd2-a6b5fdd0b816 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:57.029137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d54f8-5eaa-4452-9fd2-a6b5fdd0b816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:57.029862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d54f8-5eaa-4452-9fd2-a6b5fdd0b816 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=1eae69c100414f56ad94e32e5e70143f, user.id=02b70e4c6ee24156b986b028d366dcbc) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:57.030163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d54f8-5eaa-4452-9fd2-a6b5fdd0b816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:57.031598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d54f8-5eaa-4452-9fd2-a6b5fdd0b816 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:57.037888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 594/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:44:56 2022] GET /identity/v3/role_assignments?user.id=02b70e4c6ee24156b986b028d366dcbc&scope.project.id=1eae69c100414f56ad94e32e5e70143f&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:58.888597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db7ebf-659e-4253-ab5a-5d3ee1414695 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:58.889139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db7ebf-659e-4253-ab5a-5d3ee1414695 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:58.889139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db7ebf-659e-4253-ab5a-5d3ee1414695 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:58.890466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 592/1186] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:44:58.895631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3794-6f50-4fd7-85c6-f2bb90a6bc9b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:58.896041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3794-6f50-4fd7-85c6-f2bb90a6bc9b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:58.896186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3794-6f50-4fd7-85c6-f2bb90a6bc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:58.914566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-859a3794-6f50-4fd7-85c6-f2bb90a6bc9b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:44:58.915754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-859a3794-6f50-4fd7-85c6-f2bb90a6bc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:44:58.979858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 595/1187] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:44:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:44:59.250417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db7ebf-659e-4253-ab5a-5d3ee1414695 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:59.343192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:59.350242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db7ebf-659e-4253-ab5a-5d3ee1414695 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:59.382956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:59.383572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:59.383847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:59.384116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:59.393769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:44:59.399029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:44:59.400122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:44:59.401174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 593/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:44:59.406230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a3794-6f50-4fd7-85c6-f2bb90a6bc9b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:59.408408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:59.502395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a3794-6f50-4fd7-85c6-f2bb90a6bc9b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:59.535320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:59.535909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:59.536184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:59.536481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:59.537132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:44:59.537430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:59.538814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:59.546573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 596/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:59.552721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:59.647657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03277046-2c98-4e4c-9e69-8edc27f14f48 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:59.679775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:59.680416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:59.680695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:59.680969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] PATH_INFO: `/v3/users/02b70e4c6ee24156b986b028d366dcbc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:59.682063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] RBAC: Authorizing `identity:get_user(user_id=02b70e4c6ee24156b986b028d366dcbc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:59.683479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:59.685042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 594/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:44:59 2022] GET /identity/v3/users/02b70e4c6ee24156b986b028d366dcbc => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:59.690502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:59.750341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:59.786261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293b83-9b20-4426-9093-65e25d798623 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:59.818175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dbffa-b157-41c5-9a12-6cfeb737a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:59.818779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dbffa-b157-41c5-9a12-6cfeb737a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:59.819052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dbffa-b157-41c5-9a12-6cfeb737a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:59.819321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dbffa-b157-41c5-9a12-6cfeb737a6c7 None admin] PATH_INFO: `/v3/projects/1eae69c100414f56ad94e32e5e70143f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:59.820430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dbffa-b157-41c5-9a12-6cfeb737a6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=1eae69c100414f56ad94e32e5e70143f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:59.821844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dbffa-b157-41c5-9a12-6cfeb737a6c7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:59.823412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 597/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:44:59 2022] GET /identity/v3/projects/1eae69c100414f56ad94e32e5e70143f => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:44:59.829155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:44:59.853218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:44:59.925396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d001b-fb2f-42f1-ad7e-ff5563bfd096 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:44:59.958295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30116068-c6dd-4387-a681-aff3aa4a6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:44:59.958928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30116068-c6dd-4387-a681-aff3aa4a6f50 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:44:59.959206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30116068-c6dd-4387-a681-aff3aa4a6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:44:59.959481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30116068-c6dd-4387-a681-aff3aa4a6f50 None admin] PATH_INFO: `/v3/projects/1eae69c100414f56ad94e32e5e70143f/users/02b70e4c6ee24156b986b028d366dcbc/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:44:59.961357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30116068-c6dd-4387-a681-aff3aa4a6f50 None admin] RBAC: Authorizing `identity:create_grant(project_id=1eae69c100414f56ad94e32e5e70143f, user_id=02b70e4c6ee24156b986b028d366dcbc, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:44:59.962752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30116068-c6dd-4387-a681-aff3aa4a6f50 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:44:59.974524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 595/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:44:59 2022] PUT /identity/v3/projects/1eae69c100414f56ad94e32e5e70143f/users/02b70e4c6ee24156b986b028d366dcbc/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:45:01.816449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542bdd9-943d-420c-a3ea-d6096ff74358 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:01.817026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542bdd9-943d-420c-a3ea-d6096ff74358 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:01.817026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542bdd9-943d-420c-a3ea-d6096ff74358 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:01.818325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 598/1193] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:45:01.823285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde5388-c29f-4b50-8210-2fb5e53fdf63 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:01.823716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde5388-c29f-4b50-8210-2fb5e53fdf63 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:01.823832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde5388-c29f-4b50-8210-2fb5e53fdf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:01.841708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dcde5388-c29f-4b50-8210-2fb5e53fdf63 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:45:01.842946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dcde5388-c29f-4b50-8210-2fb5e53fdf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:45:01.907774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 596/1194] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:45:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:45:02.178598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542bdd9-943d-420c-a3ea-d6096ff74358 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:02.275258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542bdd9-943d-420c-a3ea-d6096ff74358 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:02.303022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:02.308384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:02.309007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:02.309283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:02.309552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:02.320233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:45:02.325002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:45:02.326081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:45:02.327121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 599/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:45:02.332185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde5388-c29f-4b50-8210-2fb5e53fdf63 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:02.349924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:02.430335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde5388-c29f-4b50-8210-2fb5e53fdf63 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:02.463062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:02.463656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:02.463933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:02.464210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:02.464894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:45:02.465193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:02.466540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:02.474201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 597/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:45:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:45:02.480039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:02.575259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033279f9-6d71-4c32-bbb0-e371683b5228 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:02.607023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:02.607628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:02.607902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:02.608171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] PATH_INFO: `/v3/users/02b70e4c6ee24156b986b028d366dcbc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:02.609303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] RBAC: Authorizing `identity:get_user(user_id=02b70e4c6ee24156b986b028d366dcbc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:02.610722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:02.612509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 600/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Tue Aug 30 01:45:02 2022] GET /identity/v3/users/02b70e4c6ee24156b986b028d366dcbc => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:45:02.618172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:02.692692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:02.714473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a77c6a-68d3-46ed-a1de-3939af2d0619 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:02.746443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:02.747024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:02.747303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:02.747574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:02.811100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:02.812553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:02.884293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:02.907055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 598/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:45:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:45:02.912899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:02.969616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:03.009626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39952881-37f1-40b1-a5db-b0d274a4260d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:03.041072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709e77-662f-4388-81ce-38ee89c5b205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:03.041679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709e77-662f-4388-81ce-38ee89c5b205 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:03.041950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709e77-662f-4388-81ce-38ee89c5b205 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:03.042218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709e77-662f-4388-81ce-38ee89c5b205 None admin] PATH_INFO: `/v3/projects/1eae69c100414f56ad94e32e5e70143f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:03.043306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709e77-662f-4388-81ce-38ee89c5b205 None admin] RBAC: Authorizing `identity:get_project(project_id=1eae69c100414f56ad94e32e5e70143f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:03.044748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709e77-662f-4388-81ce-38ee89c5b205 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:03.046320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 601/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Tue Aug 30 01:45:02 2022] GET /identity/v3/projects/1eae69c100414f56ad94e32e5e70143f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:45:03.052854 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:03.145789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab7ae-6f83-46c0-b54f-b41b9ec45d42 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:03.178901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa977caa-8788-416c-9e71-78f228cebe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:03.179477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa977caa-8788-416c-9e71-78f228cebe09 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:03.179751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa977caa-8788-416c-9e71-78f228cebe09 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:03.180020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa977caa-8788-416c-9e71-78f228cebe09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:03.180751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa977caa-8788-416c-9e71-78f228cebe09 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=1eae69c100414f56ad94e32e5e70143f, user.id=02b70e4c6ee24156b986b028d366dcbc) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:45:03.181053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa977caa-8788-416c-9e71-78f228cebe09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:03.182437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa977caa-8788-416c-9e71-78f228cebe09 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:03.187859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 599/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:45:03 2022] GET /identity/v3/role_assignments?user.id=02b70e4c6ee24156b986b028d366dcbc&scope.project.id=1eae69c100414f56ad94e32e5e70143f&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:45:13.037058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=36 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:13.185394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=80 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:58.114141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234b21-fd7f-47b5-b25d-2bb675329239 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:58.114735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234b21-fd7f-47b5-b25d-2bb675329239 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:58.114735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234b21-fd7f-47b5-b25d-2bb675329239 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:58.116275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 602/1201] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:45:58.121920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce11f-c88b-4c9a-b559-7ea24cc66798 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:58.122445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce11f-c88b-4c9a-b559-7ea24cc66798 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:58.122445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce11f-c88b-4c9a-b559-7ea24cc66798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:58.142040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-14cce11f-c88b-4c9a-b559-7ea24cc66798 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:45:58.143249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14cce11f-c88b-4c9a-b559-7ea24cc66798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:45:58.244338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 600/1202] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:45:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:45:58.558642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f234b21-fd7f-47b5-b25d-2bb675329239 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:58.698416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f234b21-fd7f-47b5-b25d-2bb675329239 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:58.746058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:58.746992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:58.747422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:58.747834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:58.763600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:58.765228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:45:58.787723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:45:58.788893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:45:58.790248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:45:58.790248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:45:58.790248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:45:58.790248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 603/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:45:58.795908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cce11f-c88b-4c9a-b559-7ea24cc66798 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:58.931502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cce11f-c88b-4c9a-b559-7ea24cc66798 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:58.969649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:58.977441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8161c-55d1-4b21-a572-ea49b411adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:58.978108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8161c-55d1-4b21-a572-ea49b411adef None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:58.978365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8161c-55d1-4b21-a572-ea49b411adef None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:58.978658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8161c-55d1-4b21-a572-ea49b411adef None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:58.979347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8161c-55d1-4b21-a572-ea49b411adef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:45:58.979651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8161c-55d1-4b21-a572-ea49b411adef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:58.981079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8161c-55d1-4b21-a572-ea49b411adef None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:58.994499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 601/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:45:59.000584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:45:59.063170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:45:59.125380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7aff41-18e8-44a7-8a13-97ee1219732c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:45:59.169667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f745288-f658-4bfa-b9de-fea5d99b030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:45:59.170505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f745288-f658-4bfa-b9de-fea5d99b030d None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:45:59.170901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f745288-f658-4bfa-b9de-fea5d99b030d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:45:59.171269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f745288-f658-4bfa-b9de-fea5d99b030d None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:45:59.172038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f745288-f658-4bfa-b9de-fea5d99b030d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:45:59.173641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f745288-f658-4bfa-b9de-fea5d99b030d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:45:59.203819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 604/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Tue Aug 30 01:45:58 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:46:01.270821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a704-8c5a-4f2d-ab81-7456d39d6e83 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:01.271398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a704-8c5a-4f2d-ab81-7456d39d6e83 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:01.271398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a704-8c5a-4f2d-ab81-7456d39d6e83 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:01.272778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 602/1206] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:01.278712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f700d-671e-4170-826b-033a9f339072 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:01.279234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f700d-671e-4170-826b-033a9f339072 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:01.279349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f700d-671e-4170-826b-033a9f339072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:01.303965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-787f700d-671e-4170-826b-033a9f339072 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:01.305205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-787f700d-671e-4170-826b-033a9f339072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:01.389103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 605/1207] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:46:01.660399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023a704-8c5a-4f2d-ab81-7456d39d6e83 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:01.783135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023a704-8c5a-4f2d-ab81-7456d39d6e83 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:01.824377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:01.825013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:01.825291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:01.825570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:01.840285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:46:01.845256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:01.846205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:01.847092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 603/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:46:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:01.853084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f700d-671e-4170-826b-033a9f339072 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:01.895605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:01.957328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f700d-671e-4170-826b-033a9f339072 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:01.993075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:01.993668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:01.993941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:01.994241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:01.994876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:01.995173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:01.996613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:02.005156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 606/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:46:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:02.011062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:02.066885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:02.133105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa1cc1-dcc3-4419-a8fe-f597948084d4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:02.180721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:02.181379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:02.181779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:02.182091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:02.196458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:02.197940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:02.216987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:02.218009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:02.218906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:02.218906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:02.218906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:02.218906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 604/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:02.224037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:02.329855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:02.334299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cceb9-37bf-4073-a596-e27f1e2a42fa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:02.371334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:02.371932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:02.372207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:02.372575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:02.373220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:02.373517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:02.374871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:02.387580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 607/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:02.393120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:02.505444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355f05-994a-434f-9356-9c3a3646b7c6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:02.507030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:02.542177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:02.542813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:02.543092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:02.543359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:02.551847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:02.553376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:46:02.564889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:02.567348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:02.568335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 605/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:02.572860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:02.702308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78ec25-e73f-4d04-a9da-c7c036e95945 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:02.748429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:02.749206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:02.749573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:02.749938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:02.750789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:02.751173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:02.752802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:02.774543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 608/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Tue Aug 30 01:46:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:02.781332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:02.912232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4353-3c6f-4bb2-8231-337b94432002 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:02.950719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:02.951559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:02.951957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:02.952353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:02.998026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:03.042256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:03.043986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:03.166432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 606/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:46:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:46:03.172071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:03.192041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:03.300367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7d374-489d-400b-a6dc-62eb23fb2e5c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:03.341593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:03.342342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:03.342594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:03.342898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:03.356631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:03.358468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:03.376919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:03.377897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:03.378790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:03.378790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:03.378790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:03.380824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 609/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:03.386935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:03.450554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:03.492254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eaea4-9192-406a-a506-26678a0d5927 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:03.533640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:03.534307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:03.535095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:03.535358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:03.536079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:03.536338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:03.537757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:03.549830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 607/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:46:03.555772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:03.639685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:03.689644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcefcee-56a3-4a3d-84d6-377bfb650ca5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:03.729314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:03.729977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:03.730257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:03.730536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:03.739167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:03.741354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:46:03.757429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:03.758838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:03.759888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 610/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:03.764847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:03.881903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9bb6d-4770-426d-81a3-d3569cf49e2b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:03.908692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:03.922254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf30afd-1082-469b-99a1-649a6a14c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:03.923009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf30afd-1082-469b-99a1-649a6a14c053 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:03.923352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf30afd-1082-469b-99a1-649a6a14c053 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:03.923689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf30afd-1082-469b-99a1-649a6a14c053 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:03.924494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf30afd-1082-469b-99a1-649a6a14c053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:03.925447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf30afd-1082-469b-99a1-649a6a14c053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:03.926930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf30afd-1082-469b-99a1-649a6a14c053 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:03.939547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 608/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:46:03.946588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:04.060515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6618-b730-41a9-a7c9-71dc7d86cf33 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:04.094739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:04.102271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c7ee5-7d69-4f02-8148-482f201e7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:04.102883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c7ee5-7d69-4f02-8148-482f201e7037 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:04.103209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c7ee5-7d69-4f02-8148-482f201e7037 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:04.103619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c7ee5-7d69-4f02-8148-482f201e7037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:04.104418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c7ee5-7d69-4f02-8148-482f201e7037 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=588e9ed6789e46d6a6693cb54e6fdce1) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:04.104804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c7ee5-7d69-4f02-8148-482f201e7037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:04.106529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c7ee5-7d69-4f02-8148-482f201e7037 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:04.124270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 611/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:46:03 2022] GET /identity/v3/role_assignments?user.id=588e9ed6789e46d6a6693cb54e6fdce1&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:06.085321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2e2c-54dd-488b-ab5e-2b1446590193 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:06.085928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2e2c-54dd-488b-ab5e-2b1446590193 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:06.086074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2e2c-54dd-488b-ab5e-2b1446590193 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:06.087924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 609/1220] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:06.093294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc04e0-0d24-48cc-949b-147eddf13c17 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:06.093762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc04e0-0d24-48cc-949b-147eddf13c17 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:06.093860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc04e0-0d24-48cc-949b-147eddf13c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:06.112752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2edc04e0-0d24-48cc-949b-147eddf13c17 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:06.114286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2edc04e0-0d24-48cc-949b-147eddf13c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:06.181531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 612/1221] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:46:06.474410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2e2c-54dd-488b-ab5e-2b1446590193 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:06.582572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2e2c-54dd-488b-ab5e-2b1446590193 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:06.617458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:06.618094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:06.618377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:06.618652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:06.629673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:46:06.634458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:06.635589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:06.636605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 610/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:46:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:06.641691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc04e0-0d24-48cc-949b-147eddf13c17 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:06.743649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc04e0-0d24-48cc-949b-147eddf13c17 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:06.771473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:06.783681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:06.784893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:06.785163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:06.785431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:06.786253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:06.786555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:06.788050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:06.796072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 613/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:46:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:46:06.802636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:06.852506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:06.912124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0438fa-7334-4936-ae09-2c90a3b8b420 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:06.947639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:06.948307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:06.948584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:06.948852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:06.958683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:06.960133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:06.979563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:06.980837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:06.981722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:06.981722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:06.981722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:06.981722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 611/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:06.986726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:07.101128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc3641-b6b1-4061-b7df-43ed7a811770 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:07.137694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:07.138307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:07.138584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:07.138853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:07.139489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:07.139788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:07.141181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:07.155226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 614/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:07.162241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:07.262012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:07.270188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9329afd-2c87-4d41-835b-0abda7a588b9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:07.305935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:07.306571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:07.306855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:07.307132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:07.315233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:07.316897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:46:07.329326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:07.330528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:07.332346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 612/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:46:07.337328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:07.387188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:07.442218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b9ad3-cc78-4793-b0c8-5f692bd6a20d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:07.477832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:07.478870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:07.479272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:07.479717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:07.480680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:07.481019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:07.482671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:07.503101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 615/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Tue Aug 30 01:46:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:46:07.508953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:07.616225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f825c-286f-4687-bf52-11d032335e32 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:07.649710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:07.650253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:07.650548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:07.650812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:07.662135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:07.663554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:07.679301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:07.680657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:07.681836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:07.681836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:07.681836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:07.681836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 613/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:07.686452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:07.805427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f858bc-1682-4348-9db0-b6b4535db1b0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:07.808831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:07.855632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:07.856622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:07.857055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:07.857482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:07.858372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:07.858736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:07.861032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:07.881272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 616/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:07.882687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:07.895265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:07.995307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe936c15-f0a6-497e-a46c-66ea5f5ed21f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:08.031012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:08.031631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:08.031905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:08.032182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:08.038969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:08.040431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:46:08.053713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:08.054805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:08.055733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 614/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:46:08.061355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:08.167912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c492b0-ff29-4652-baf0-f29ce6ec9991 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:08.204992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcd637-f77b-44f9-a91c-91b64a826095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:08.205609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcd637-f77b-44f9-a91c-91b64a826095 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:08.205888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcd637-f77b-44f9-a91c-91b64a826095 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:08.206193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcd637-f77b-44f9-a91c-91b64a826095 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:08.206881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcd637-f77b-44f9-a91c-91b64a826095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:08.207188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcd637-f77b-44f9-a91c-91b64a826095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:08.208664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcd637-f77b-44f9-a91c-91b64a826095 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:08.220643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 617/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:08.226929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:08.296155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:08.335587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72f5c0-3c45-4c46-b291-1cbbc2b88587 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:08.371223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494686-19f8-4314-9996-2afcfa02f98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:08.371872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494686-19f8-4314-9996-2afcfa02f98b None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:08.372150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494686-19f8-4314-9996-2afcfa02f98b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:08.372450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494686-19f8-4314-9996-2afcfa02f98b None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/588e9ed6789e46d6a6693cb54e6fdce1/roles/3aa65bdcfd1244cb9d358fa12c4c02cc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:08.389102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494686-19f8-4314-9996-2afcfa02f98b None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=588e9ed6789e46d6a6693cb54e6fdce1, role_id=3aa65bdcfd1244cb9d358fa12c4c02cc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:08.391393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494686-19f8-4314-9996-2afcfa02f98b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:08.403723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 615/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:46:08 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/588e9ed6789e46d6a6693cb54e6fdce1/roles/3aa65bdcfd1244cb9d358fa12c4c02cc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:46:10.293954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337a24-002d-48d1-a94d-ed73ccb88bca None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:10.294466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337a24-002d-48d1-a94d-ed73ccb88bca None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:10.294567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337a24-002d-48d1-a94d-ed73ccb88bca None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:10.295872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 618/1233] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:10.301128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f67be-31b2-44ef-b0cc-785085b4f90a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:10.301521 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f67be-31b2-44ef-b0cc-785085b4f90a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:10.301677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f67be-31b2-44ef-b0cc-785085b4f90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:10.321859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-484f67be-31b2-44ef-b0cc-785085b4f90a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:10.323499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-484f67be-31b2-44ef-b0cc-785085b4f90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:10.396806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 616/1234] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:46:10.695414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337a24-002d-48d1-a94d-ed73ccb88bca None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:10.726384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:10.820799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337a24-002d-48d1-a94d-ed73ccb88bca None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:10.872676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:10.873514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:10.873984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:10.874287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:10.888972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:46:10.895280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:10.896939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:10.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 619/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Tue Aug 30 01:46:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:10.902937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f67be-31b2-44ef-b0cc-785085b4f90a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:10.953746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:11.018930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f67be-31b2-44ef-b0cc-785085b4f90a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:11.053695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:11.054305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:11.054584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:11.054857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:11.055511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:11.055811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:11.057221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:11.065104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 617/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Tue Aug 30 01:46:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:46:11.071186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:11.164109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:11.185608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cf809-5afd-4558-8cf7-00645ded049a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:11.224861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:11.225420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:11.225703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:11.225977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:11.239363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:11.241269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:11.259122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:11.261752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:11.263686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:11.263686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:11.263686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:11.266270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 620/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:11.267325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:11.383994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcf0d2-e3f9-4c0e-ba5a-213bb6d056bd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:11.393239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:11.422672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:11.423258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:11.423536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:11.423838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:11.424548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:11.424857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:11.426264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:11.439049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 618/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:11.445305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:11.552660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607339-06ae-4f9c-babd-ae39598656ef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:11.594016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:11.594915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:11.595213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:11.595481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:11.599086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:11.606558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:11.608795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:46:11.625191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:11.626546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:11.628488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 621/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:46:11.634983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:11.760623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0760f4c-7e0b-4768-a341-2e13721e4422 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:11.804770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:11.805313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:11.805673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:11.806014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:11.807030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:11.807336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:11.809116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:11.829013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 619/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Tue Aug 30 01:46:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:11.836860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:11.966527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e9cfe-9988-40c9-9aa6-c88c4b43e4dd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:12.005925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:12.006537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:12.006823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:12.007088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:12.070793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:12.077490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:12.078951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:12.184056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 622/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:46:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:46:12.189832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:12.220638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:12.296233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0197da-feec-463a-8428-45c743e5b1ac None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:12.333232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:12.333793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:12.334068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:12.334336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:12.344462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:12.345901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:12.361104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:12.363147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:12.364226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:12.364226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:12.364226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:12.364226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 620/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:12.368908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:12.470850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:12.478585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57583-cc8f-454f-bbdb-9cb7f8be040f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:12.515159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:12.515767 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:12.516043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:12.516404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:12.517194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:12.517510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:12.519624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:12.537569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 623/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:12.538879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:12.621718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:12.642773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d07764-43fe-4698-8006-458320ac87d7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:12.679418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:12.680067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:12.680393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:12.680665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:12.688189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:12.689643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:46:12.701064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:12.702144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:12.703256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 621/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:46:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:12.709994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:12.824008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a8c92-32a5-4617-88aa-d946cd498092 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:12.875725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f38d4-fdf8-455b-9e26-91970eb4912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:12.876266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f38d4-fdf8-455b-9e26-91970eb4912a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:12.876555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f38d4-fdf8-455b-9e26-91970eb4912a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:12.876830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f38d4-fdf8-455b-9e26-91970eb4912a None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:12.877524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f38d4-fdf8-455b-9e26-91970eb4912a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:12.878316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f38d4-fdf8-455b-9e26-91970eb4912a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:12.879749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f38d4-fdf8-455b-9e26-91970eb4912a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:12.895501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 624/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:46:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:12.903408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:13.011973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421aee-4e78-45af-9266-2f8b8b76d5bf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:13.052036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52d9d2-d31d-4c8b-8c17-7bb7c1559299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:13.052668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52d9d2-d31d-4c8b-8c17-7bb7c1559299 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:13.052950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52d9d2-d31d-4c8b-8c17-7bb7c1559299 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:13.053224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52d9d2-d31d-4c8b-8c17-7bb7c1559299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:13.053938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52d9d2-d31d-4c8b-8c17-7bb7c1559299 None admin] RBAC: Adding query filter params (role.id=3aa65bdcfd1244cb9d358fa12c4c02cc, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=588e9ed6789e46d6a6693cb54e6fdce1) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:13.054246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52d9d2-d31d-4c8b-8c17-7bb7c1559299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:13.055665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52d9d2-d31d-4c8b-8c17-7bb7c1559299 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:13.061506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 622/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:46:12 2022] GET /identity/v3/role_assignments?user.id=588e9ed6789e46d6a6693cb54e6fdce1&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=3aa65bdcfd1244cb9d358fa12c4c02cc => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:46:15.277620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc02bb-5fcd-4078-9f8f-727e7d100f0b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:15.278167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc02bb-5fcd-4078-9f8f-727e7d100f0b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:15.278167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc02bb-5fcd-4078-9f8f-727e7d100f0b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:15.279499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 625/1247] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:15.285733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b88cc4-a245-470b-ae0c-05769911042e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:15.286253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b88cc4-a245-470b-ae0c-05769911042e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:15.286253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b88cc4-a245-470b-ae0c-05769911042e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:15.294442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:15.308394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-88b88cc4-a245-470b-ae0c-05769911042e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:15.309971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-88b88cc4-a245-470b-ae0c-05769911042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:15.386529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 623/1248] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 01:46:15.654759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc02bb-5fcd-4078-9f8f-727e7d100f0b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:15.673208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:15.763799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc02bb-5fcd-4078-9f8f-727e7d100f0b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:15.801232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:15.801858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:15.802138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:15.802409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:15.816391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:46:15.825211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:15.826509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:15.827518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 626/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:46:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:15.832969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b88cc4-a245-470b-ae0c-05769911042e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:15.966806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b88cc4-a245-470b-ae0c-05769911042e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:15.996547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:16.005955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:16.006657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:16.007004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:16.007283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:16.007943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:16.008271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:16.009646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:16.018009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 624/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:46:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:46:16.023973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:16.110827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:16.140861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315f03-32d0-472f-9747-192c5e65d7e3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:16.177140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:16.177771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:16.178044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:16.178314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:16.190467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:16.191921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:16.207985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:16.209909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:16.211192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:16.211192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:16.211192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:16.211192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 627/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:46:16.215358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:16.326704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32759a-22cf-4e40-9dcb-b0ffd51d8507 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:16.370427 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:16.371089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:16.371440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:16.371767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:16.372553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:16.372917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:16.374483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:16.387300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 625/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:16.394983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:16.498255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b9ba0-02c1-4c9f-a1bc-699904bc6dd6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:16.526240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:16.535391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:16.536024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:16.536358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:16.536628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:16.544448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:16.546097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:46:16.557897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:16.559080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:16.560222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 628/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 01:46:16.567924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:16.631993 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:16.679025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ecae6-830e-4d88-950c-6978924ffc72 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:16.715766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:16.716448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:16.716726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:16.717001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:16.717759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:16.718093 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:16.719616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:16.737849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 626/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Tue Aug 30 01:46:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:16.744751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:16.858161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2757e0-d9c9-4aa9-a277-5ff4a2631f2d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:16.900023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:16.900722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:16.901002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:16.901278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:16.964332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:16.978991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:16.980496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:17.090354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 629/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:46:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 01:46:17.097456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:17.214937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc82c8-2aad-45c7-b68a-41e4964c6b42 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:17.230135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:17.254816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:17.255467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:17.255745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:17.256016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:17.266985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:17.268485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:17.283663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:17.285437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:17.286697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:17.286697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:17.286697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:17.286697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 627/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:17.291860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:17.385943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:17.399900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619cf7-0b29-4f23-baec-8197b1ef2346 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:17.438761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:17.439398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:17.439677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:17.439976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:17.441203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:17.441508 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:17.442931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:17.455291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 630/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:17.462147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:17.569684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfa64b-e0be-4f06-8822-44df260b41cc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:17.604537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:17.605177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:17.605453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:17.605720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:17.613817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:17.615292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:46:17.627312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:17.628472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:17.629494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 628/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:46:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:17.634724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:17.739070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51eaeef-d2b2-494f-825a-c5178a271122 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:17.774889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb71b9a-15b9-442c-9df5-17a15821965a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:17.775514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb71b9a-15b9-442c-9df5-17a15821965a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:17.775796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb71b9a-15b9-442c-9df5-17a15821965a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:17.776072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb71b9a-15b9-442c-9df5-17a15821965a None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:17.776818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb71b9a-15b9-442c-9df5-17a15821965a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:17.777126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb71b9a-15b9-442c-9df5-17a15821965a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:17.778578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb71b9a-15b9-442c-9df5-17a15821965a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:17.792427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 631/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:17.798584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:17.833626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:17.924063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487ace0-5273-4656-9988-79421e0b2ab6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:17.957629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74846102-4d8c-4b99-baa7-337e1a67ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:17.958241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74846102-4d8c-4b99-baa7-337e1a67ed71 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:17.958519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74846102-4d8c-4b99-baa7-337e1a67ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:17.958790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74846102-4d8c-4b99-baa7-337e1a67ed71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:17.959510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74846102-4d8c-4b99-baa7-337e1a67ed71 None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=588e9ed6789e46d6a6693cb54e6fdce1) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:17.959814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74846102-4d8c-4b99-baa7-337e1a67ed71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:17.961342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74846102-4d8c-4b99-baa7-337e1a67ed71 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:17.966715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 629/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:46:17 2022] GET /identity/v3/role_assignments?user.id=588e9ed6789e46d6a6693cb54e6fdce1&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:19.828194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e2d25-d57f-4a0d-bc29-17b0810cd6bb None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:19.828744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e2d25-d57f-4a0d-bc29-17b0810cd6bb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:19.828744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e2d25-d57f-4a0d-bc29-17b0810cd6bb None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:19.830108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 632/1261] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:19.836683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab5a29-bf08-406f-b7ff-8f47aa2e359a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:19.837236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab5a29-bf08-406f-b7ff-8f47aa2e359a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:19.837351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab5a29-bf08-406f-b7ff-8f47aa2e359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:19.859837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-13ab5a29-bf08-406f-b7ff-8f47aa2e359a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:19.861128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-13ab5a29-bf08-406f-b7ff-8f47aa2e359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:19.930494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 630/1262] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:46:20.200445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e2d25-d57f-4a0d-bc29-17b0810cd6bb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:20.211553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:20.299677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e2d25-d57f-4a0d-bc29-17b0810cd6bb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:20.333186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:20.333806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:20.334089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:20.334362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:20.344020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:46:20.348825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:20.349897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:20.351010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 633/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:46:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:20.355824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab5a29-bf08-406f-b7ff-8f47aa2e359a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:20.373533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:20.450633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab5a29-bf08-406f-b7ff-8f47aa2e359a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:20.482672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:20.483254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:20.483528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:20.483799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:20.484469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:20.484768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:20.486103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:20.493977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 631/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:46:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:20.499665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:20.564209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:20.597175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cd2f6-19e3-4d29-a572-dbace4c1570b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:20.629514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:20.630121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:20.630413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:20.630671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:20.640696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:20.642137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:20.656363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:20.657465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:20.658546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:20.658546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:20.658546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:20.658546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 634/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:20.663170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:20.740599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:20.759313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce27bcf-07e6-4099-9d0b-a51c64b4cb8f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:20.790389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:20.790970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:20.791244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:20.791511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:20.792129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:20.792453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:20.793787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:20.805886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 632/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:20.811486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:20.918285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837e32d-30cf-4fdd-9e99-6683d7e7aa72 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:20.937209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:20.951442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:20.952060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:20.952371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:20.952643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:20.959723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:20.961189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:46:20.971985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:20.973074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:20.974060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 635/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:20.978958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:21.074061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8e969-ce11-4aab-9b27-392d0c81bd32 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:21.106101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:21.106689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:21.106965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:21.107237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:21.107948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:21.108275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:21.109672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:21.118733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:21.126813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 633/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Tue Aug 30 01:46:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:21.132357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:21.228033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fa3bd-dcc5-4a2a-8e69-2a5b42ec97c9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:21.260781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:21.261393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:21.261669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:21.261940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:21.271944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:21.273419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:21.287530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:21.288470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:21.289350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:21.289350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:21.289350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:21.289350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 636/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:21.294567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:21.389686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b368c5-50f8-4273-b368-0d5a12c3fbdc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:21.421710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:21.422300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:21.422577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:21.422846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:21.423474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:21.423774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:21.425709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:21.436520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 634/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:21.442088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:21.473411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:21.539591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3c5d1-8b00-4052-9095-c72c657f53b4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:21.571448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:21.572049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:21.572353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:21.572628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:21.578989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:21.580436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:46:21.591793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:21.592859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:21.594079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 637/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:21.599166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:21.656508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:21.697345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77349a-cf5b-4600-bfb2-861eb9f9160d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:21.730354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf2252-355f-44fc-99dc-01f965c6480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:21.730945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2252-355f-44fc-99dc-01f965c6480c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:21.731224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2252-355f-44fc-99dc-01f965c6480c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:21.731499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2252-355f-44fc-99dc-01f965c6480c None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:21.732178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf2252-355f-44fc-99dc-01f965c6480c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:21.732511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf2252-355f-44fc-99dc-01f965c6480c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:21.733965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf2252-355f-44fc-99dc-01f965c6480c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:21.746048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 635/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:21.751893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:21.843088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:21.853575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b09d3c-35c0-489f-b4df-b80a9ab78d50 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:21.892721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b3fb6-2221-4ee2-b668-2514af06e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:21.894001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b3fb6-2221-4ee2-b668-2514af06e5e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:21.894273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b3fb6-2221-4ee2-b668-2514af06e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:21.894543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b3fb6-2221-4ee2-b668-2514af06e5e6 None admin] PATH_INFO: `/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/588e9ed6789e46d6a6693cb54e6fdce1/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:21.896902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b3fb6-2221-4ee2-b668-2514af06e5e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=eba2b9510c624752808c81bcd11a5f54, user_id=588e9ed6789e46d6a6693cb54e6fdce1, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:21.898321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b3fb6-2221-4ee2-b668-2514af06e5e6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:21.910609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 638/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Tue Aug 30 01:46:21 2022] PUT /identity/v3/projects/eba2b9510c624752808c81bcd11a5f54/users/588e9ed6789e46d6a6693cb54e6fdce1/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:46:23.811764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e306c0f-63a7-48e3-8f61-c084cfc5965d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:23.812346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e306c0f-63a7-48e3-8f61-c084cfc5965d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:23.812346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e306c0f-63a7-48e3-8f61-c084cfc5965d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:23.813691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 636/1274] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:23.818863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc890-4d48-4d92-ac5d-888ecc8ce4d7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:23.819238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc890-4d48-4d92-ac5d-888ecc8ce4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:23.819407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc890-4d48-4d92-ac5d-888ecc8ce4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:23.837542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5b5cc890-4d48-4d92-ac5d-888ecc8ce4d7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:23.838705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5b5cc890-4d48-4d92-ac5d-888ecc8ce4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:23.909660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 639/1275] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:46:24.182358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e306c0f-63a7-48e3-8f61-c084cfc5965d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:24.281346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e306c0f-63a7-48e3-8f61-c084cfc5965d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:24.298550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:24.315464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:24.316081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:24.316387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:24.316654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:24.326228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 01:46:24.331542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:24.332675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:24.333516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 637/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Tue Aug 30 01:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:24.338553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cc890-4d48-4d92-ac5d-888ecc8ce4d7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:24.415219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:24.434248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cc890-4d48-4d92-ac5d-888ecc8ce4d7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:24.466321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:24.466913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:24.467188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:24.467456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:24.468105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:24.468431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:24.469783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:24.477351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 640/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Tue Aug 30 01:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:24.482875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:24.578688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59e430-4e3c-43dc-8b6d-10d2c4814f13 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:24.611176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:24.611776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:24.612071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:24.612362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:24.622024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:24.623429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:24.637185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:24.638249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:24.639510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:24.639510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:24.639510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:24.639510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 638/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:24.644300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:24.738216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89223fe-91ad-403f-b470-3ac16b79c177 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:24.769659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:24.770237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:24.770509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:24.770777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:24.771404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:24.771696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:24.773066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:24.784018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 641/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:24.789424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:24.819040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:24.896730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5f4a7-287e-4b32-8452-13c28188ee13 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:24.931022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:24.931640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:24.931920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:24.932193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:24.939353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:24.941317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 01:46:24.952159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:24.953132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:24.954118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 639/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:24.959861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:24.963577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:25.055856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff54c-bff8-47e7-80e2-144958881030 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:25.087337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:25.087921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:25.088194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:25.088507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:25.089221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:25.089524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:25.090937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:25.107538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 642/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Tue Aug 30 01:46:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:25.113671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:25.201348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:25.211791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846db6ff-9f9a-44d9-a321-2d52d7ad705b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:25.245170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:25.245764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:25.246043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:25.246315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:25.310794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:25.312189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:25.395778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:25.406216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 640/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Tue Aug 30 01:46:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 01:46:25.412593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:25.476537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:25.513135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf314f-4724-4d7b-8523-7f64ca8e3726 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:25.546125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:25.546743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:25.547018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:25.547288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:25.556733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:25.558172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 01:46:25.572606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:25.573699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 01:46:25.574993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 01:46:25.574993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 01:46:25.574993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:25.574993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 643/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:25.579371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:25.673981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7546-3e41-4cb7-aa6f-360a3ae38a19 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:25.707518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:25.708106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:25.708418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:25.708687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:25.709311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:25.709607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:25.710949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:25.721680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 641/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Tue Aug 30 01:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:25.727121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:25.822754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e1e42-ec61-4156-a3b2-5964c2092e6a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:25.836208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:25.855643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:25.856260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:25.856557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:25.856828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:25.863421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:25.864885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 01:46:25.875717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:25.876735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Aug 30 01:46:25.878325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 644/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Tue Aug 30 01:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:25.883004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:25.932977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:25.986705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e0ff4-6987-4b1a-b04e-b7b36946829c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:26.020564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b9a7f-5e8c-41d3-a62c-a4b839b3d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:26.021190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b9a7f-5e8c-41d3-a62c-a4b839b3d163 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:26.021489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b9a7f-5e8c-41d3-a62c-a4b839b3d163 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:26.021780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b9a7f-5e8c-41d3-a62c-a4b839b3d163 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:26.022522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b9a7f-5e8c-41d3-a62c-a4b839b3d163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:26.022853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b9a7f-5e8c-41d3-a62c-a4b839b3d163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:26.024447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b9a7f-5e8c-41d3-a62c-a4b839b3d163 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:26.036762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 642/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Tue Aug 30 01:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:26.042824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:26.139779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb61ab-f07c-4f62-9f88-00ec95449123 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:26.172464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60d3b0-f47b-4173-9841-1c2d35be1bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:26.173070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60d3b0-f47b-4173-9841-1c2d35be1bcd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:26.173350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60d3b0-f47b-4173-9841-1c2d35be1bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:26.173621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60d3b0-f47b-4173-9841-1c2d35be1bcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:26.174853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60d3b0-f47b-4173-9841-1c2d35be1bcd None admin] RBAC: Adding query filter params (role.id=78859ba2c4f645bea8c7e9d0fa00c5aa, scope.project.id=eba2b9510c624752808c81bcd11a5f54, user.id=588e9ed6789e46d6a6693cb54e6fdce1) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:26.175160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60d3b0-f47b-4173-9841-1c2d35be1bcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:26.176606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60d3b0-f47b-4173-9841-1c2d35be1bcd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:26.182109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 645/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Tue Aug 30 01:46:26 2022] GET /identity/v3/role_assignments?user.id=588e9ed6789e46d6a6693cb54e6fdce1&scope.project.id=eba2b9510c624752808c81bcd11a5f54&role.id=78859ba2c4f645bea8c7e9d0fa00c5aa => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:28.063582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd1026-0f43-4135-9a53-b29f69ffba70 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:28.064125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd1026-0f43-4135-9a53-b29f69ffba70 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:28.064125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd1026-0f43-4135-9a53-b29f69ffba70 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:28.065494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 643/1288] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:28.070834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9da1-099c-4f72-a5b3-6ada57976936 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:28.071229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9da1-099c-4f72-a5b3-6ada57976936 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:28.071387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9da1-099c-4f72-a5b3-6ada57976936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:28.089388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-13ad9da1-099c-4f72-a5b3-6ada57976936 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:28.090607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13ad9da1-099c-4f72-a5b3-6ada57976936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:28.120258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:28.155833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 646/1289] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:46:28.434436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd1026-0f43-4135-9a53-b29f69ffba70 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:28.532778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd1026-0f43-4135-9a53-b29f69ffba70 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:28.543606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:28.566801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:28.567419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:28.567697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:28.567970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:28.568570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:28.569923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:28.578403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:28.579642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:28.580946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 644/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:46:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:28.585431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad9da1-099c-4f72-a5b3-6ada57976936 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:28.679171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad9da1-099c-4f72-a5b3-6ada57976936 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:28.710582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc899f1e-2ea1-48af-b982-7f49e2e2d69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:28.711112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899f1e-2ea1-48af-b982-7f49e2e2d69a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:28.711383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899f1e-2ea1-48af-b982-7f49e2e2d69a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:28.711653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899f1e-2ea1-48af-b982-7f49e2e2d69a None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:28.712288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc899f1e-2ea1-48af-b982-7f49e2e2d69a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:28.712590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc899f1e-2ea1-48af-b982-7f49e2e2d69a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:28.713940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc899f1e-2ea1-48af-b982-7f49e2e2d69a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:28.718490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 647/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Tue Aug 30 01:46:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:28.724607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:28.820222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21323d-18a2-44cf-b154-a1eeef2891a4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:28.854899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07207af-ecb6-4b27-a1fc-0b7477dbd61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:28.855437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07207af-ecb6-4b27-a1fc-0b7477dbd61e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:28.855712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07207af-ecb6-4b27-a1fc-0b7477dbd61e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:28.855985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07207af-ecb6-4b27-a1fc-0b7477dbd61e None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:28.856640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07207af-ecb6-4b27-a1fc-0b7477dbd61e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:28.856943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07207af-ecb6-4b27-a1fc-0b7477dbd61e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:28.858301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07207af-ecb6-4b27-a1fc-0b7477dbd61e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:28.864307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 645/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Tue Aug 30 01:46:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:30.732820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a96e8-4171-48ea-8670-b24d848806b6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:30.733331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a96e8-4171-48ea-8670-b24d848806b6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:30.733431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a96e8-4171-48ea-8670-b24d848806b6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:30.734673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 648/1293] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:30.740603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3f93-c47b-47ee-b83c-f6d5d52d51ea None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:30.740990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3f93-c47b-47ee-b83c-f6d5d52d51ea None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:30.741155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3f93-c47b-47ee-b83c-f6d5d52d51ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:30.759487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4f6e3f93-c47b-47ee-b83c-f6d5d52d51ea None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:30.760700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4f6e3f93-c47b-47ee-b83c-f6d5d52d51ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:30.789398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:30.826658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 646/1294] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:46:31.091111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a96e8-4171-48ea-8670-b24d848806b6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:31.121413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:31.190916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a96e8-4171-48ea-8670-b24d848806b6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:31.222668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4067329-6660-472d-b86c-7dd2e2288f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:31.223256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4067329-6660-472d-b86c-7dd2e2288f2c None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:31.223531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4067329-6660-472d-b86c-7dd2e2288f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:31.223802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4067329-6660-472d-b86c-7dd2e2288f2c None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:31.224472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4067329-6660-472d-b86c-7dd2e2288f2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:31.225819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4067329-6660-472d-b86c-7dd2e2288f2c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:31.239990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 649/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Tue Aug 30 01:46:31 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:46:33.102853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b0145-067d-45a0-8814-a39ec379d358 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:33.103411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b0145-067d-45a0-8814-a39ec379d358 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:33.103411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b0145-067d-45a0-8814-a39ec379d358 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:33.104793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 647/1296] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:33.109848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c370f51-d3cd-4d5c-a90f-473b56c45e56 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:33.110254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c370f51-d3cd-4d5c-a90f-473b56c45e56 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:33.110391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c370f51-d3cd-4d5c-a90f-473b56c45e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:33.128875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2c370f51-d3cd-4d5c-a90f-473b56c45e56 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:33.130048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2c370f51-d3cd-4d5c-a90f-473b56c45e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:33.198894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 650/1297] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:46:33.464614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b0145-067d-45a0-8814-a39ec379d358 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:33.561719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b0145-067d-45a0-8814-a39ec379d358 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:33.593991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:33.594602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:33.594879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:33.595149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:33.595725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:33.597738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:33.606174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:33.607153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:33.608052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 648/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:46:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:33.613438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c370f51-d3cd-4d5c-a90f-473b56c45e56 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:33.631137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:33.708541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c370f51-d3cd-4d5c-a90f-473b56c45e56 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:33.740975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:33.741571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:33.741846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:33.742118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:33.742738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:33.743038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:33.744454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:33.749335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 651/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Tue Aug 30 01:46:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:33.755227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:33.786990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:33.854531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d62f81-3497-46d4-a77f-54f6659a5932 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:33.886701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f20bf-ca88-425f-9632-206dd5993d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:33.887299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f20bf-ca88-425f-9632-206dd5993d44 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:33.887577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f20bf-ca88-425f-9632-206dd5993d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:33.887849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f20bf-ca88-425f-9632-206dd5993d44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:33.888573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f20bf-ca88-425f-9632-206dd5993d44 None admin] RBAC: Adding query filter params (interface=public, service_id=17bc3805720a4951921860a693d99e76, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:33.888879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f20bf-ca88-425f-9632-206dd5993d44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:33.890226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f20bf-ca88-425f-9632-206dd5993d44 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:33.895663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 649/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Tue Aug 30 01:46:33 2022] GET /identity/v3/endpoints?service_id=17bc3805720a4951921860a693d99e76&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:33.901433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:33.979042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:33.997964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4e042-fdbc-4bb0-94bb-dbaa9b1a849e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:34.031413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b427c67-fda4-49f3-bffb-0f5d502fd8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:34.032008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b427c67-fda4-49f3-bffb-0f5d502fd8bc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:34.032308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b427c67-fda4-49f3-bffb-0f5d502fd8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:34.032583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b427c67-fda4-49f3-bffb-0f5d502fd8bc None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:34.033181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b427c67-fda4-49f3-bffb-0f5d502fd8bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:34.035065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b427c67-fda4-49f3-bffb-0f5d502fd8bc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:34.041572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 652/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Tue Aug 30 01:46:33 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:46:35.901232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5013-dabf-4508-b764-ad774ff9484e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:35.901780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5013-dabf-4508-b764-ad774ff9484e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:35.901780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5013-dabf-4508-b764-ad774ff9484e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:35.903107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 650/1302] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:46:35.908209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29df7-4a98-4a06-a413-b6bcffb09663 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:35.908646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29df7-4a98-4a06-a413-b6bcffb09663 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:35.908787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29df7-4a98-4a06-a413-b6bcffb09663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:35.926809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d4a29df7-4a98-4a06-a413-b6bcffb09663 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:46:35.927963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d4a29df7-4a98-4a06-a413-b6bcffb09663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:46:35.991540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 653/1303] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:46:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 01:46:36.255828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5013-dabf-4508-b764-ad774ff9484e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:36.347327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:36.354142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5013-dabf-4508-b764-ad774ff9484e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:36.386399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:36.387014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:36.387288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:36.387558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] PATH_INFO: `/v3/services/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:36.388126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:36.389501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 01:46:36.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 01:46:36.399078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 30 01:46:36.399914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 651/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Tue Aug 30 01:46:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 01:46:36.404923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29df7-4a98-4a06-a413-b6bcffb09663 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:36.477178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:36.499327 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29df7-4a98-4a06-a413-b6bcffb09663 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:36.532349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b679af2-26be-4850-9193-d87f39f1ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:36.532884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679af2-26be-4850-9193-d87f39f1ffee None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:36.533156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679af2-26be-4850-9193-d87f39f1ffee None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:36.533442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679af2-26be-4850-9193-d87f39f1ffee None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:36.534591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b679af2-26be-4850-9193-d87f39f1ffee None admin] RBAC: Adding query filter params (name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:46:36.534890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b679af2-26be-4850-9193-d87f39f1ffee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:36.536264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b679af2-26be-4850-9193-d87f39f1ffee None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:36.541089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 654/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Tue Aug 30 01:46:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:46:36.546663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:46:36.641533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9555-667a-4931-a186-edd289284cac None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:46:36.674699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee506e-05cb-4dd7-806d-346b02dbbb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:46:36.675289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee506e-05cb-4dd7-806d-346b02dbbb2c None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:46:36.675566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee506e-05cb-4dd7-806d-346b02dbbb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:46:36.675839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee506e-05cb-4dd7-806d-346b02dbbb2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:46:36.676537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee506e-05cb-4dd7-806d-346b02dbbb2c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:46:36.677880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee506e-05cb-4dd7-806d-346b02dbbb2c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:46:36.696450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 652/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:46:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:46:46.538303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=30 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:46:46.692512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:39.869389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2fc59-0fda-472b-a94d-aa90ac09763e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:39.869796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2fc59-0fda-472b-a94d-aa90ac09763e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:39.869889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2fc59-0fda-472b-a94d-aa90ac09763e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:39.871256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 655/1307] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:39.877008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c77fa-ef29-43eb-ae07-cc9e6d809e09 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:39.877408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c77fa-ef29-43eb-ae07-cc9e6d809e09 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:39.877408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c77fa-ef29-43eb-ae07-cc9e6d809e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:39.965275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-101c77fa-ef29-43eb-ae07-cc9e6d809e09 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:39.966463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-101c77fa-ef29-43eb-ae07-cc9e6d809e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:40.094431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 653/1308] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:40.389602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527653-53e4-40f2-8ab0-79c629eb3ad9 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:40.390354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527653-53e4-40f2-8ab0-79c629eb3ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:40.390865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527653-53e4-40f2-8ab0-79c629eb3ad9 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:40.393085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 656/1309] 10.222.0.50 () {54 vars in 965 bytes} [Tue Aug 30 01:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:40.402914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264feca6-ab0a-4f54-9b3b-0e52dd635a08 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:40.403279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264feca6-ab0a-4f54-9b3b-0e52dd635a08 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:40.403471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264feca6-ab0a-4f54-9b3b-0e52dd635a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:40.496106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-264feca6-ab0a-4f54-9b3b-0e52dd635a08 None None] MFA Rules not processed for user `e1c3b7f7c83f48aca27fbd46202f6506`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:40.497922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-264feca6-ab0a-4f54-9b3b-0e52dd635a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:40.520441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:40.583285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 654/1310] 10.222.0.50 () {58 vars in 1062 bytes} [Tue Aug 30 01:47:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:47:40.592781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527653-53e4-40f2-8ab0-79c629eb3ad9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:47:40.594846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d9527653-53e4-40f2-8ab0-79c629eb3ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:40.713882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527653-53e4-40f2-8ab0-79c629eb3ad9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:47:40.793781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f5d73-8e3e-423e-8d08-0a79f83b357b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:47:40.794388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f5d73-8e3e-423e-8d08-0a79f83b357b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:40.794666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f5d73-8e3e-423e-8d08-0a79f83b357b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:40.794939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f5d73-8e3e-423e-8d08-0a79f83b357b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:40.796697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ad7f5d73-8e3e-423e-8d08-0a79f83b357b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:40.856506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:40.915588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f5d73-8e3e-423e-8d08-0a79f83b357b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:47:40.917154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f5d73-8e3e-423e-8d08-0a79f83b357b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:47:40.936888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 657/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:47:43.199039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec2fa9-10e3-448d-a6ac-c6bfbe68cf39 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:43.199571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec2fa9-10e3-448d-a6ac-c6bfbe68cf39 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:43.199571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec2fa9-10e3-448d-a6ac-c6bfbe68cf39 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:43.204293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 655/1312] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:43.207000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bd5f1-8ffa-4a41-8b6e-621919ff44c9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:43.207463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bd5f1-8ffa-4a41-8b6e-621919ff44c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:43.207717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bd5f1-8ffa-4a41-8b6e-621919ff44c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:43.289709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0f7bd5f1-8ffa-4a41-8b6e-621919ff44c9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:43.291283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0f7bd5f1-8ffa-4a41-8b6e-621919ff44c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:43.394654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 658/1313] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:43.676716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec2fa9-10e3-448d-a6ac-c6bfbe68cf39 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:47:43.678291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-98ec2fa9-10e3-448d-a6ac-c6bfbe68cf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:43.780479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec2fa9-10e3-448d-a6ac-c6bfbe68cf39 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:47:43.850725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:43.853358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8aea87-6233-4867-b7c4-1f300b4ba4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:47:43.854245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aea87-6233-4867-b7c4-1f300b4ba4eb service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:43.854580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aea87-6233-4867-b7c4-1f300b4ba4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:43.855016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aea87-6233-4867-b7c4-1f300b4ba4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:43.856809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fa8aea87-6233-4867-b7c4-1f300b4ba4eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:43.962502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8aea87-6233-4867-b7c4-1f300b4ba4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:47:43.963913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8aea87-6233-4867-b7c4-1f300b4ba4eb service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:47:43.982167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 656/1314] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:47:46.171051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d986e-3fd0-4d2d-bf1d-e8295ddea606 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:46.171609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d986e-3fd0-4d2d-bf1d-e8295ddea606 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:46.171609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d986e-3fd0-4d2d-bf1d-e8295ddea606 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:46.173552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 659/1315] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:46.178547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12eb993-8fe1-4d62-aca9-f429f256b6f2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:46.178984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12eb993-8fe1-4d62-aca9-f429f256b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:46.179094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12eb993-8fe1-4d62-aca9-f429f256b6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:46.242574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e12eb993-8fe1-4d62-aca9-f429f256b6f2 None None] MFA Rules not processed for user `801bea5ab5ed40fd9053f989885b4314`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:46.243766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e12eb993-8fe1-4d62-aca9-f429f256b6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:46.294042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:46.337814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 657/1316] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:47:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:46.618692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d986e-3fd0-4d2d-bf1d-e8295ddea606 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:47:46.620212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bf8d986e-3fd0-4d2d-bf1d-e8295ddea606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:46.630175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:46.723543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d986e-3fd0-4d2d-bf1d-e8295ddea606 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:47:46.797706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120102b9-7a9e-4f41-b518-bca994da52c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:47:46.798825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120102b9-7a9e-4f41-b518-bca994da52c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:46.799102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120102b9-7a9e-4f41-b518-bca994da52c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:46.799370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120102b9-7a9e-4f41-b518-bca994da52c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:46.800983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-120102b9-7a9e-4f41-b518-bca994da52c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:46.879034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:46.928929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120102b9-7a9e-4f41-b518-bca994da52c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:47:46.930368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120102b9-7a9e-4f41-b518-bca994da52c4 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:47:46.948261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 660/1317] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:47:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:47:49.368271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6beb80-6483-4aae-9e73-4a5c773fd1d1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:49.368768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6beb80-6483-4aae-9e73-4a5c773fd1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:49.368892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6beb80-6483-4aae-9e73-4a5c773fd1d1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:49.370143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 658/1318] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:49.375156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9c0b-6568-4fad-8450-a17282350bad None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:49.375560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9c0b-6568-4fad-8450-a17282350bad None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:49.375701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9c0b-6568-4fad-8450-a17282350bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:49.439271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c14f9c0b-6568-4fad-8450-a17282350bad None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:49.440502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c14f9c0b-6568-4fad-8450-a17282350bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:49.535040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 661/1319] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:47:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:49.815615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6beb80-6483-4aae-9e73-4a5c773fd1d1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:47:49.817178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8c6beb80-6483-4aae-9e73-4a5c773fd1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:49.923773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6beb80-6483-4aae-9e73-4a5c773fd1d1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:47:49.994719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a2625-45f3-414b-8b3d-1ae9d6ffddb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:47:49.995317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2625-45f3-414b-8b3d-1ae9d6ffddb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:49.995594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2625-45f3-414b-8b3d-1ae9d6ffddb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:49.995868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a2625-45f3-414b-8b3d-1ae9d6ffddb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:49.997482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b1a2625-45f3-414b-8b3d-1ae9d6ffddb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:50.005267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:50.103567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a2625-45f3-414b-8b3d-1ae9d6ffddb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:47:50.105003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a2625-45f3-414b-8b3d-1ae9d6ffddb3 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:47:50.124582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 659/1320] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:47:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:47:52.650775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56078f-3545-4297-aaf4-0ae2e501c1df None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:52.651283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56078f-3545-4297-aaf4-0ae2e501c1df None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:52.651283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56078f-3545-4297-aaf4-0ae2e501c1df None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:52.652644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 662/1321] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:52.657829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221ca3b-ed06-4fc2-a015-2bef54f51fc6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:52.658235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221ca3b-ed06-4fc2-a015-2bef54f51fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:52.658378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221ca3b-ed06-4fc2-a015-2bef54f51fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:52.723041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5221ca3b-ed06-4fc2-a015-2bef54f51fc6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:52.724260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5221ca3b-ed06-4fc2-a015-2bef54f51fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:52.780129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:52.821237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 660/1322] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:53.107274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56078f-3545-4297-aaf4-0ae2e501c1df None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:47:53.109028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3b56078f-3545-4297-aaf4-0ae2e501c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:53.130964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:53.212863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56078f-3545-4297-aaf4-0ae2e501c1df None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:47:53.288480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6b9bd-662e-47eb-a7d4-e93c3e22bd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:47:53.289093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6b9bd-662e-47eb-a7d4-e93c3e22bd72 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:53.289371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6b9bd-662e-47eb-a7d4-e93c3e22bd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:53.289644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6b9bd-662e-47eb-a7d4-e93c3e22bd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:53.291305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-30d6b9bd-662e-47eb-a7d4-e93c3e22bd72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:53.364929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:53.401543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6b9bd-662e-47eb-a7d4-e93c3e22bd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:47:53.402990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6b9bd-662e-47eb-a7d4-e93c3e22bd72 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:47:53.421083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 663/1323] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:47:55.370711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520c968-a491-42ff-915a-f8d615a78940 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:55.371235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520c968-a491-42ff-915a-f8d615a78940 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:55.371355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520c968-a491-42ff-915a-f8d615a78940 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:55.372657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 661/1324] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:55.378421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9927-4729-4972-9009-326a8ed1ce99 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:55.378849 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9927-4729-4972-9009-326a8ed1ce99 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:55.378981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9927-4729-4972-9009-326a8ed1ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:55.444199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-783a9927-4729-4972-9009-326a8ed1ce99 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:55.445338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-783a9927-4729-4972-9009-326a8ed1ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:55.540143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 664/1325] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:55.838489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b13c-f260-4d18-87ab-36bcf9638ac4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:55.838877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b13c-f260-4d18-87ab-36bcf9638ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:55.839225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b13c-f260-4d18-87ab-36bcf9638ac4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:55.840755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 662/1326] 10.222.0.50 () {54 vars in 965 bytes} [Tue Aug 30 01:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:55.848427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1eef6-0727-4f4d-98fb-a3c86c7988ac None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:55.848710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1eef6-0727-4f4d-98fb-a3c86c7988ac None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:55.848982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1eef6-0727-4f4d-98fb-a3c86c7988ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:55.881325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:55.927558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a8f1eef6-0727-4f4d-98fb-a3c86c7988ac None None] MFA Rules not processed for user `e1c3b7f7c83f48aca27fbd46202f6506`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:55.928782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a8f1eef6-0727-4f4d-98fb-a3c86c7988ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:56.005159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 665/1327] 10.222.0.50 () {58 vars in 1062 bytes} [Tue Aug 30 01:47:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:56.014311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0b13c-f260-4d18-87ab-36bcf9638ac4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:47:56.015802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-44c0b13c-f260-4d18-87ab-36bcf9638ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:56.129002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0b13c-f260-4d18-87ab-36bcf9638ac4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:47:56.200966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2984a-c302-4eb1-a4bd-79f170c79842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:47:56.201569 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2984a-c302-4eb1-a4bd-79f170c79842 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:56.201841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2984a-c302-4eb1-a4bd-79f170c79842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:56.202107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2984a-c302-4eb1-a4bd-79f170c79842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:56.203671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dea2984a-c302-4eb1-a4bd-79f170c79842 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:56.219978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:56.314588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2984a-c302-4eb1-a4bd-79f170c79842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:47:56.315944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2984a-c302-4eb1-a4bd-79f170c79842 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:47:56.333470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 663/1328] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:47:58.528669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130068-59ca-4b2a-b3f6-a1c8e6b7f551 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:58.529180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130068-59ca-4b2a-b3f6-a1c8e6b7f551 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:58.529281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130068-59ca-4b2a-b3f6-a1c8e6b7f551 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:58.530594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 666/1329] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:47:58.536157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e950d-4fe8-4601-addd-bcff9bf11605 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:58.536621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e950d-4fe8-4601-addd-bcff9bf11605 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:58.536760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e950d-4fe8-4601-addd-bcff9bf11605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:58.617987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c56e950d-4fe8-4601-addd-bcff9bf11605 None None] MFA Rules not processed for user `801bea5ab5ed40fd9053f989885b4314`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:47:58.619278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c56e950d-4fe8-4601-addd-bcff9bf11605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:58.686676 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:58.722537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 664/1330] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:47:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:47:59.009748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130068-59ca-4b2a-b3f6-a1c8e6b7f551 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:47:59.011267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-74130068-59ca-4b2a-b3f6-a1c8e6b7f551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:59.113846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130068-59ca-4b2a-b3f6-a1c8e6b7f551 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:47:59.133831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:47:59.189259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae989dc3-59a0-4cd8-8851-f110edee12ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:47:59.189866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae989dc3-59a0-4cd8-8851-f110edee12ce service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:47:59.190139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae989dc3-59a0-4cd8-8851-f110edee12ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:47:59.190406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae989dc3-59a0-4cd8-8851-f110edee12ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:47:59.191994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ae989dc3-59a0-4cd8-8851-f110edee12ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:47:59.305784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae989dc3-59a0-4cd8-8851-f110edee12ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:47:59.307066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae989dc3-59a0-4cd8-8851-f110edee12ce service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:47:59.328634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 667/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:47:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:01.469627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53a01-b3d3-45b2-a86c-0bf272f5ea81 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:01.470200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53a01-b3d3-45b2-a86c-0bf272f5ea81 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:01.470200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53a01-b3d3-45b2-a86c-0bf272f5ea81 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:01.471494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 665/1332] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:01.476908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b3428-8e07-4738-b372-1878fb8ae609 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:01.477334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b3428-8e07-4738-b372-1878fb8ae609 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:01.477455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b3428-8e07-4738-b372-1878fb8ae609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:01.539792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:01.542907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cd2b3428-8e07-4738-b372-1878fb8ae609 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:01.544638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cd2b3428-8e07-4738-b372-1878fb8ae609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:01.641681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 668/1333] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:01.920345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab53a01-b3d3-45b2-a86c-0bf272f5ea81 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:01.921947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6ab53a01-b3d3-45b2-a86c-0bf272f5ea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:02.022989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab53a01-b3d3-45b2-a86c-0bf272f5ea81 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:02.093290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4e9b9-b308-49bc-bb1c-64daed633d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:02.093887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4e9b9-b308-49bc-bb1c-64daed633d82 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:02.094702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4e9b9-b308-49bc-bb1c-64daed633d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:02.094971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4e9b9-b308-49bc-bb1c-64daed633d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:02.096563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c2c4e9b9-b308-49bc-bb1c-64daed633d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:02.122908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:02.202732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4e9b9-b308-49bc-bb1c-64daed633d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:02.204100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4e9b9-b308-49bc-bb1c-64daed633d82 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:02.221904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 666/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:04.154449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f463c-d5a4-4093-be79-7c084c3830e3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:04.155043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f463c-d5a4-4093-be79-7c084c3830e3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:04.155043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f463c-d5a4-4093-be79-7c084c3830e3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:04.156604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 669/1335] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:04.162079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a5e2-c1f2-4a1e-84b5-5cc0a1a14526 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:04.162468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a5e2-c1f2-4a1e-84b5-5cc0a1a14526 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:04.162630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a5e2-c1f2-4a1e-84b5-5cc0a1a14526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:04.226820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3f45a5e2-c1f2-4a1e-84b5-5cc0a1a14526 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:04.228310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3f45a5e2-c1f2-4a1e-84b5-5cc0a1a14526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:04.300448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:04.324051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 667/1336] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:04.609877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f463c-d5a4-4093-be79-7c084c3830e3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:04.611402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-206f463c-d5a4-4093-be79-7c084c3830e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:04.711996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f463c-d5a4-4093-be79-7c084c3830e3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:04.740915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:04.785197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9364445-87bd-4b4d-83c6-d63354528204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:04.785782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9364445-87bd-4b4d-83c6-d63354528204 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:04.786053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9364445-87bd-4b4d-83c6-d63354528204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:04.786318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9364445-87bd-4b4d-83c6-d63354528204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:04.787866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b9364445-87bd-4b4d-83c6-d63354528204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:04.892950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9364445-87bd-4b4d-83c6-d63354528204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:04.894306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9364445-87bd-4b4d-83c6-d63354528204 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:04.912257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 670/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:07.115087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d494-35fd-4193-a2d5-e4f680c12691 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:07.115607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d494-35fd-4193-a2d5-e4f680c12691 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:07.115607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d494-35fd-4193-a2d5-e4f680c12691 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:07.116953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 668/1338] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:07.122049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0aaad-61b3-4086-8c3a-026198bd25b8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:07.122448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0aaad-61b3-4086-8c3a-026198bd25b8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:07.122598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0aaad-61b3-4086-8c3a-026198bd25b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:07.186697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c7e0aaad-61b3-4086-8c3a-026198bd25b8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:07.188417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c7e0aaad-61b3-4086-8c3a-026198bd25b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:07.196380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:07.281946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 671/1339] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:07.558434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028d494-35fd-4193-a2d5-e4f680c12691 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:07.559986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d028d494-35fd-4193-a2d5-e4f680c12691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:07.660979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028d494-35fd-4193-a2d5-e4f680c12691 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:07.734998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7cbf6-0e0d-4590-9084-30fc04a1147d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:07.735593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7cbf6-0e0d-4590-9084-30fc04a1147d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:07.735869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7cbf6-0e0d-4590-9084-30fc04a1147d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:07.736141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7cbf6-0e0d-4590-9084-30fc04a1147d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:07.737739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-92f7cbf6-0e0d-4590-9084-30fc04a1147d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:07.772180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:07.842930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7cbf6-0e0d-4590-9084-30fc04a1147d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:07.844875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7cbf6-0e0d-4590-9084-30fc04a1147d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:07.862412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 669/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:11.663714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125429-ce2f-4027-8c23-2e3494584968 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:11.664309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125429-ce2f-4027-8c23-2e3494584968 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:11.664309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32125429-ce2f-4027-8c23-2e3494584968 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:11.665666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 672/1341] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:11.670886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2de68-56b5-4527-90b1-99ee8748b704 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:11.671298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2de68-56b5-4527-90b1-99ee8748b704 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:11.671417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2de68-56b5-4527-90b1-99ee8748b704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:11.738974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4fb2de68-56b5-4527-90b1-99ee8748b704 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:11.740526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4fb2de68-56b5-4527-90b1-99ee8748b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:11.823628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:11.835759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 670/1342] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:12.124997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125429-ce2f-4027-8c23-2e3494584968 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:12.126516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-32125429-ce2f-4027-8c23-2e3494584968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:12.229278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32125429-ce2f-4027-8c23-2e3494584968 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:12.265556 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:12.305820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ec800-2731-4716-a175-8c6d2089877f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:12.306420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ec800-2731-4716-a175-8c6d2089877f service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:12.306694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ec800-2731-4716-a175-8c6d2089877f service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:12.306962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ec800-2731-4716-a175-8c6d2089877f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:12.308883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d78ec800-2731-4716-a175-8c6d2089877f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:12.416543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ec800-2731-4716-a175-8c6d2089877f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:12.417902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ec800-2731-4716-a175-8c6d2089877f service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:12.435798 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 673/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:14.999170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688435c-bc97-4ab9-aff5-e85b407ed55b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:14.999704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688435c-bc97-4ab9-aff5-e85b407ed55b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:14.999704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688435c-bc97-4ab9-aff5-e85b407ed55b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:15.001780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 671/1344] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:15.006787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce848be-dbe2-4693-bd17-3aebdda825e7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:15.007271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce848be-dbe2-4693-bd17-3aebdda825e7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:15.007714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce848be-dbe2-4693-bd17-3aebdda825e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:15.079842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7ce848be-dbe2-4693-bd17-3aebdda825e7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:15.081077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7ce848be-dbe2-4693-bd17-3aebdda825e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:15.098772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:15.179122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 674/1345] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:15.464270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688435c-bc97-4ab9-aff5-e85b407ed55b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:15.465791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7688435c-bc97-4ab9-aff5-e85b407ed55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:15.570986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688435c-bc97-4ab9-aff5-e85b407ed55b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:15.646428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b366b7-6ee1-4f7a-894e-e86871d3700c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:15.647037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b366b7-6ee1-4f7a-894e-e86871d3700c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:15.647308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b366b7-6ee1-4f7a-894e-e86871d3700c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:15.647593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b366b7-6ee1-4f7a-894e-e86871d3700c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:15.649251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-13b366b7-6ee1-4f7a-894e-e86871d3700c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:15.692254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:15.756841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b366b7-6ee1-4f7a-894e-e86871d3700c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:15.758255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b366b7-6ee1-4f7a-894e-e86871d3700c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:15.776168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 672/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:19.687017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bd914-b84b-4930-83bb-915732ad0541 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:19.688131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bd914-b84b-4930-83bb-915732ad0541 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:19.688285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bd914-b84b-4930-83bb-915732ad0541 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:19.691526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 675/1347] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:19.695125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7b1b-6d44-4a00-bf6d-3bd9a9563bb8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:19.695539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7b1b-6d44-4a00-bf6d-3bd9a9563bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:19.695711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa7b1b-6d44-4a00-bf6d-3bd9a9563bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:19.771793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2eaa7b1b-6d44-4a00-bf6d-3bd9a9563bb8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:19.773200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2eaa7b1b-6d44-4a00-bf6d-3bd9a9563bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:19.884339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:19.888503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 673/1348] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:20.173491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bd914-b84b-4930-83bb-915732ad0541 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:20.175059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e43bd914-b84b-4930-83bb-915732ad0541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:20.287931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bd914-b84b-4930-83bb-915732ad0541 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:20.336163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:20.361666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf71bce-fda5-459f-8b64-52a0f02a90a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:20.362264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71bce-fda5-459f-8b64-52a0f02a90a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:20.362538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71bce-fda5-459f-8b64-52a0f02a90a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:20.362808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf71bce-fda5-459f-8b64-52a0f02a90a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:20.364681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cdf71bce-fda5-459f-8b64-52a0f02a90a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:20.470939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf71bce-fda5-459f-8b64-52a0f02a90a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:20.472405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf71bce-fda5-459f-8b64-52a0f02a90a7 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:20.490063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 676/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:25.518854 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5ff53-0750-4426-b880-8a70eb846a39 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:25.519363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5ff53-0750-4426-b880-8a70eb846a39 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:25.519472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5ff53-0750-4426-b880-8a70eb846a39 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:25.521309 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 674/1350] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 01:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:25.527779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3a0d9-cbd3-49f1-ae9d-0a9e13893f10 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:25.528835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3a0d9-cbd3-49f1-ae9d-0a9e13893f10 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:25.529313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3a0d9-cbd3-49f1-ae9d-0a9e13893f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:25.654497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0de3a0d9-cbd3-49f1-ae9d-0a9e13893f10 None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:25.655689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0de3a0d9-cbd3-49f1-ae9d-0a9e13893f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:25.710318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:25.822687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 677/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 01:48:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:29.949342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62f43a-8209-4694-aef7-91f3cc5eec78 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:29.949928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62f43a-8209-4694-aef7-91f3cc5eec78 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:29.949928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62f43a-8209-4694-aef7-91f3cc5eec78 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:29.951130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 675/1352] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 01:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:29.956105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8ebe-3810-4af1-945f-01884b255f8b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:29.956571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8ebe-3810-4af1-945f-01884b255f8b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:29.956701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8ebe-3810-4af1-945f-01884b255f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:30.039080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dffe8ebe-3810-4af1-945f-01884b255f8b None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:30.040629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dffe8ebe-3810-4af1-945f-01884b255f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:30.117907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:30.158682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 678/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 01:48:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:31.562136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb695-552d-4e42-8a37-0bbd6ec46712 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:31.562657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb695-552d-4e42-8a37-0bbd6ec46712 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:31.562657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb695-552d-4e42-8a37-0bbd6ec46712 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:31.564003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 676/1354] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 01:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:31.569186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b17e1-afe8-4a6b-8450-c6ef8515c0f8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:31.569600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b17e1-afe8-4a6b-8450-c6ef8515c0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:31.569854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b17e1-afe8-4a6b-8450-c6ef8515c0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:31.646966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-739b17e1-afe8-4a6b-8450-c6ef8515c0f8 None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:31.648164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-739b17e1-afe8-4a6b-8450-c6ef8515c0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:31.755717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 679/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 01:48:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:35.092888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fe2e5-1802-4fc1-aa0d-29174a315a00 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:35.093917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fe2e5-1802-4fc1-aa0d-29174a315a00 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:35.094112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fe2e5-1802-4fc1-aa0d-29174a315a00 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:35.095572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 677/1356] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:35.101250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40435174-86db-411f-9867-113015edadc1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:35.102053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40435174-86db-411f-9867-113015edadc1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:35.102328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40435174-86db-411f-9867-113015edadc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:35.155316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:35.193859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-40435174-86db-411f-9867-113015edadc1 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:35.195204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40435174-86db-411f-9867-113015edadc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:35.327476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 680/1357] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:35.631865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d80e6-c356-4c8e-b30e-57d8b00bf65b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:35.632661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d80e6-c356-4c8e-b30e-57d8b00bf65b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:35.633167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d80e6-c356-4c8e-b30e-57d8b00bf65b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:35.635434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 678/1358] 10.222.0.50 () {54 vars in 962 bytes} [Tue Aug 30 01:48:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:35.648714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab4f3d-d5d4-4f41-9b4c-2a3f4828e4a4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:35.648990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab4f3d-d5d4-4f41-9b4c-2a3f4828e4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:35.649268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab4f3d-d5d4-4f41-9b4c-2a3f4828e4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:35.747568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-44ab4f3d-d5d4-4f41-9b4c-2a3f4828e4a4 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:35.748901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-44ab4f3d-d5d4-4f41-9b4c-2a3f4828e4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:35.799757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:35.872826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 681/1359] 10.222.0.50 () {58 vars in 1059 bytes} [Tue Aug 30 01:48:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:35.881576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d80e6-c356-4c8e-b30e-57d8b00bf65b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:35.883177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1b5d80e6-c356-4c8e-b30e-57d8b00bf65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:36.031168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d80e6-c356-4c8e-b30e-57d8b00bf65b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:36.111029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:36.153267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abfbb9-8d30-412e-aff5-ae935f0e1923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:36.154041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abfbb9-8d30-412e-aff5-ae935f0e1923 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:36.154418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abfbb9-8d30-412e-aff5-ae935f0e1923 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:36.154785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abfbb9-8d30-412e-aff5-ae935f0e1923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:36.157193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-60abfbb9-8d30-412e-aff5-ae935f0e1923 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:36.251512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abfbb9-8d30-412e-aff5-ae935f0e1923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:36.253311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abfbb9-8d30-412e-aff5-ae935f0e1923 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:36.272123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 679/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:38.408041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b2d2-a542-4070-8d2a-cb802de6743d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:38.408583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b2d2-a542-4070-8d2a-cb802de6743d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:38.408835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b2d2-a542-4070-8d2a-cb802de6743d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:38.410416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 682/1361] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:38.415640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a851-d60a-4e3f-a59a-8231ccdeadb7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:38.416051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a851-d60a-4e3f-a59a-8231ccdeadb7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:38.416220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6a851-d60a-4e3f-a59a-8231ccdeadb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:38.486104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2aa6a851-d60a-4e3f-a59a-8231ccdeadb7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:38.487291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2aa6a851-d60a-4e3f-a59a-8231ccdeadb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:38.525620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:38.600400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 680/1362] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:38.892386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b2d2-a542-4070-8d2a-cb802de6743d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:38.893871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6074b2d2-a542-4070-8d2a-cb802de6743d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:39.081586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b2d2-a542-4070-8d2a-cb802de6743d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:39.091911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:39.216792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bacf5-555f-4605-9afd-c0671341adba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:39.217406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bacf5-555f-4605-9afd-c0671341adba service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:39.217688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bacf5-555f-4605-9afd-c0671341adba service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:39.217957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bacf5-555f-4605-9afd-c0671341adba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:39.219738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-301bacf5-555f-4605-9afd-c0671341adba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:39.317823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bacf5-555f-4605-9afd-c0671341adba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:39.319952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bacf5-555f-4605-9afd-c0671341adba service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:39.339786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 683/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:41.570600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9c7c3-c7b8-4ceb-9863-cd2f7e14461e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:41.571126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9c7c3-c7b8-4ceb-9863-cd2f7e14461e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:41.571126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9c7c3-c7b8-4ceb-9863-cd2f7e14461e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:41.572437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 681/1364] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:41.578368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27159f-c784-4717-856c-05777351d773 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:41.578794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27159f-c784-4717-856c-05777351d773 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:41.578885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27159f-c784-4717-856c-05777351d773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:41.595474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:41.655659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ad27159f-c784-4717-856c-05777351d773 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:41.657548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ad27159f-c784-4717-856c-05777351d773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:41.757839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 684/1365] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:42.047494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2833191-aec8-46b4-838e-1e37964b097e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:42.047781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2833191-aec8-46b4-838e-1e37964b097e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:42.048230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2833191-aec8-46b4-838e-1e37964b097e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:42.049830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 682/1366] 10.222.0.50 () {54 vars in 962 bytes} [Tue Aug 30 01:48:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:42.055588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9503ce6-82fe-44a0-bf8d-edaa76674816 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:42.055961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9503ce6-82fe-44a0-bf8d-edaa76674816 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:42.056356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9503ce6-82fe-44a0-bf8d-edaa76674816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:42.139989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e9503ce6-82fe-44a0-bf8d-edaa76674816 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:42.141532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e9503ce6-82fe-44a0-bf8d-edaa76674816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:42.177736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:42.264998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 685/1367] 10.222.0.50 () {58 vars in 1059 bytes} [Tue Aug 30 01:48:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:42.273268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2833191-aec8-46b4-838e-1e37964b097e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:42.274972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d2833191-aec8-46b4-838e-1e37964b097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:42.438929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2833191-aec8-46b4-838e-1e37964b097e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:42.463278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:42.566690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffc723-53b7-4782-a886-895618702d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:42.567519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffc723-53b7-4782-a886-895618702d1e service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:42.567812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffc723-53b7-4782-a886-895618702d1e service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:42.568089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffc723-53b7-4782-a886-895618702d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:42.569759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c0ffc723-53b7-4782-a886-895618702d1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:42.670297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffc723-53b7-4782-a886-895618702d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:42.672683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffc723-53b7-4782-a886-895618702d1e service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:42.692200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 683/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:43.270941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac121e-a48c-45a3-b636-828963608d1d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:43.271521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac121e-a48c-45a3-b636-828963608d1d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:43.271521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac121e-a48c-45a3-b636-828963608d1d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:43.273017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 686/1369] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:43.278005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db063-2a2c-4f45-950f-a61a0dc78c0b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:43.278291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db063-2a2c-4f45-950f-a61a0dc78c0b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:43.278563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db063-2a2c-4f45-950f-a61a0dc78c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:43.315983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:43.352591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-837db063-2a2c-4f45-950f-a61a0dc78c0b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:43.353791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-837db063-2a2c-4f45-950f-a61a0dc78c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:43.455347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 684/1370] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:44.041394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1306e-0340-4d45-9c10-7b6a45179a7f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:44.041394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1306e-0340-4d45-9c10-7b6a45179a7f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:44.042350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1306e-0340-4d45-9c10-7b6a45179a7f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:44.043378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 687/1371] 10.222.0.50 () {54 vars in 964 bytes} [Tue Aug 30 01:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:44.049141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0656d-7c82-456c-97ba-7286298fa6a3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:44.049601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0656d-7c82-456c-97ba-7286298fa6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:44.049698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0656d-7c82-456c-97ba-7286298fa6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:44.133917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2ef0656d-7c82-456c-97ba-7286298fa6a3 None None] MFA Rules not processed for user `b68fc66dbef44519a2212d35655bf6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:44.135138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2ef0656d-7c82-456c-97ba-7286298fa6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:44.184295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:44.220008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 685/1372] 10.222.0.50 () {58 vars in 1061 bytes} [Tue Aug 30 01:48:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:48:44.227813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1306e-0340-4d45-9c10-7b6a45179a7f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:44.229409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c7e1306e-0340-4d45-9c10-7b6a45179a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:44.342761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1306e-0340-4d45-9c10-7b6a45179a7f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:44.388515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:44.428755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fbe71-fd2d-4ad0-8383-e00d31accb3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:44.429666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fbe71-fd2d-4ad0-8383-e00d31accb3b service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:44.430290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fbe71-fd2d-4ad0-8383-e00d31accb3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:44.430717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fbe71-fd2d-4ad0-8383-e00d31accb3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:44.432907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f4fbe71-fd2d-4ad0-8383-e00d31accb3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:44.568742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fbe71-fd2d-4ad0-8383-e00d31accb3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:44.570727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fbe71-fd2d-4ad0-8383-e00d31accb3b service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:44.598365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 688/1373] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:44.706678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9435a-41c3-4c32-a4ea-065ce5a95c3e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:44.707003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9435a-41c3-4c32-a4ea-065ce5a95c3e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:44.707480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9435a-41c3-4c32-a4ea-065ce5a95c3e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:44.712768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 686/1374] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:44.713682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8515fd-00a4-4dd8-83d0-9e21dfac03e6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:44.713965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8515fd-00a4-4dd8-83d0-9e21dfac03e6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:44.714237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8515fd-00a4-4dd8-83d0-9e21dfac03e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:44.798122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2a8515fd-00a4-4dd8-83d0-9e21dfac03e6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:44.799255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2a8515fd-00a4-4dd8-83d0-9e21dfac03e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:44.820154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:44.899509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 689/1375] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:48:45.186438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9435a-41c3-4c32-a4ea-065ce5a95c3e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:45.188623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-20f9435a-41c3-4c32-a4ea-065ce5a95c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:45.339975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9435a-41c3-4c32-a4ea-065ce5a95c3e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:45.393728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:45.455199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4374c24-8a67-458d-981e-ddc21fc4e66d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:45.455723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374c24-8a67-458d-981e-ddc21fc4e66d service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:45.455999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374c24-8a67-458d-981e-ddc21fc4e66d service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:45.456290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374c24-8a67-458d-981e-ddc21fc4e66d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:45.457871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b4374c24-8a67-458d-981e-ddc21fc4e66d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:45.547402 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4374c24-8a67-458d-981e-ddc21fc4e66d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:45.549023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4374c24-8a67-458d-981e-ddc21fc4e66d service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:45.567002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 687/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:47.518021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d13b6-27a8-4209-a642-1909a0a038d3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:47.518522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d13b6-27a8-4209-a642-1909a0a038d3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:47.518645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d13b6-27a8-4209-a642-1909a0a038d3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:47.519927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 690/1377] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:47.525363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da46ff-bbdb-4923-9a23-95accf7c54e9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:47.525790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da46ff-bbdb-4923-9a23-95accf7c54e9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:47.525898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da46ff-bbdb-4923-9a23-95accf7c54e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:47.595657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f8da46ff-bbdb-4923-9a23-95accf7c54e9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:47.596871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f8da46ff-bbdb-4923-9a23-95accf7c54e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:47.603555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:47.720519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 688/1378] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:48.011835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d13b6-27a8-4209-a642-1909a0a038d3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:48.013430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-620d13b6-27a8-4209-a642-1909a0a038d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:48.210864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d13b6-27a8-4209-a642-1909a0a038d3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:48.219467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:48.337530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d4460-f74d-4ab4-84a2-11ba951168ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:48.338151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4460-f74d-4ab4-84a2-11ba951168ca service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:48.338434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4460-f74d-4ab4-84a2-11ba951168ca service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:48.338711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4460-f74d-4ab4-84a2-11ba951168ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:48.340317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-172d4460-f74d-4ab4-84a2-11ba951168ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:48.438258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d4460-f74d-4ab4-84a2-11ba951168ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:48.439855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d4460-f74d-4ab4-84a2-11ba951168ca service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:48.458818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 691/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:50.400895 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d8e-1dcf-4f6c-a974-3375914f8b15 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:50.401473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d8e-1dcf-4f6c-a974-3375914f8b15 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:50.401473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d8e-1dcf-4f6c-a974-3375914f8b15 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:50.402968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 689/1380] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:50.408661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af52367-7951-4c97-a66a-6471128d739e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:50.409067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af52367-7951-4c97-a66a-6471128d739e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:50.409232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af52367-7951-4c97-a66a-6471128d739e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:50.420606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:50.479205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1af52367-7951-4c97-a66a-6471128d739e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:50.480424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1af52367-7951-4c97-a66a-6471128d739e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:50.590474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 692/1381] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:50.886930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4697d8e-1dcf-4f6c-a974-3375914f8b15 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:50.889535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f4697d8e-1dcf-4f6c-a974-3375914f8b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:51.044996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:51.057030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4697d8e-1dcf-4f6c-a974-3375914f8b15 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:51.176775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6a004-cc78-4185-b154-f9f076cab001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:51.177306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a004-cc78-4185-b154-f9f076cab001 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:51.177580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a004-cc78-4185-b154-f9f076cab001 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:51.177850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a004-cc78-4185-b154-f9f076cab001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:51.179406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72d6a004-cc78-4185-b154-f9f076cab001 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:51.275323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6a004-cc78-4185-b154-f9f076cab001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:51.276981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6a004-cc78-4185-b154-f9f076cab001 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:51.305095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:51.320177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 690/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:53.295918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672001a-302c-46ce-afc9-4d8fe5f2e0aa None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:53.297071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672001a-302c-46ce-afc9-4d8fe5f2e0aa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:53.297071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672001a-302c-46ce-afc9-4d8fe5f2e0aa None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:53.298715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 693/1383] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:53.304759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affae0c7-e8f4-4438-a32d-6f7583d27e84 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:53.305258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affae0c7-e8f4-4438-a32d-6f7583d27e84 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:53.305436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affae0c7-e8f4-4438-a32d-6f7583d27e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:53.387181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-affae0c7-e8f4-4438-a32d-6f7583d27e84 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:53.388874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-affae0c7-e8f4-4438-a32d-6f7583d27e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:53.501444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 691/1384] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:48:53.794976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672001a-302c-46ce-afc9-4d8fe5f2e0aa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:53.796509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f672001a-302c-46ce-afc9-4d8fe5f2e0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:53.885875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:53.950901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672001a-302c-46ce-afc9-4d8fe5f2e0aa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:54.071389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd54c6-0565-4d7d-b790-305bd890e5a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:54.071992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd54c6-0565-4d7d-b790-305bd890e5a1 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:54.072449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd54c6-0565-4d7d-b790-305bd890e5a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:54.072592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd54c6-0565-4d7d-b790-305bd890e5a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:54.074181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-33dd54c6-0565-4d7d-b790-305bd890e5a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:54.127072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:54.176656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd54c6-0565-4d7d-b790-305bd890e5a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:54.178649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd54c6-0565-4d7d-b790-305bd890e5a1 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:54.201967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 694/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:56.101897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9288b-e933-49d9-979f-5b99911f5182 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:56.102391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9288b-e933-49d9-979f-5b99911f5182 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:56.102621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9288b-e933-49d9-979f-5b99911f5182 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:56.104543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 692/1386] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:56.110463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dbcda-ea28-46c3-8c2e-52495600b794 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:56.110874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dbcda-ea28-46c3-8c2e-52495600b794 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:56.111023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dbcda-ea28-46c3-8c2e-52495600b794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:56.185785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a32dbcda-ea28-46c3-8c2e-52495600b794 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:56.188410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a32dbcda-ea28-46c3-8c2e-52495600b794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:56.293481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 695/1387] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:56.691450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9288b-e933-49d9-979f-5b99911f5182 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:56.693146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ec9288b-e933-49d9-979f-5b99911f5182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:56.772593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:56.852089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9288b-e933-49d9-979f-5b99911f5182 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:56.978203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680c8ec-97d9-42cd-8db4-8ddf6de286c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:56.978735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c8ec-97d9-42cd-8db4-8ddf6de286c4 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:56.979015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c8ec-97d9-42cd-8db4-8ddf6de286c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:56.979288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c8ec-97d9-42cd-8db4-8ddf6de286c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:56.980951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1680c8ec-97d9-42cd-8db4-8ddf6de286c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:57.021856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:57.078136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680c8ec-97d9-42cd-8db4-8ddf6de286c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:57.080340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680c8ec-97d9-42cd-8db4-8ddf6de286c4 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:57.103308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 693/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:48:59.043836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afca327-1c08-4b32-9f74-4a8f24dae05a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:59.044429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afca327-1c08-4b32-9f74-4a8f24dae05a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:59.044429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afca327-1c08-4b32-9f74-4a8f24dae05a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:59.045751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 696/1389] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:48:59.051368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d737fee-cdf0-43a6-a753-494402a0215b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:59.051827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d737fee-cdf0-43a6-a753-494402a0215b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:59.051926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d737fee-cdf0-43a6-a753-494402a0215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:59.117898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2d737fee-cdf0-43a6-a753-494402a0215b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:59.119082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2d737fee-cdf0-43a6-a753-494402a0215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:59.215764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 694/1390] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:48:59.509748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afca327-1c08-4b32-9f74-4a8f24dae05a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:48:59.511387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1afca327-1c08-4b32-9f74-4a8f24dae05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:59.542343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:59.599870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a216-05c5-447b-9d4e-e660b08a2afc None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:59.600152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a216-05c5-447b-9d4e-e660b08a2afc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:59.600377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01a216-05c5-447b-9d4e-e660b08a2afc None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:59.601845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 695/1391] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:48:59.607037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bc809-b29d-459f-9ee8-f34b7a4a4f54 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:59.607326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bc809-b29d-459f-9ee8-f34b7a4a4f54 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:59.607604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bc809-b29d-459f-9ee8-f34b7a4a4f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:59.623648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:59.676095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-093bc809-b29d-459f-9ee8-f34b7a4a4f54 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:48:59.677444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-093bc809-b29d-459f-9ee8-f34b7a4a4f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:59.678970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afca327-1c08-4b32-9f74-4a8f24dae05a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:48:59.776381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 696/1392] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:48:59.781515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:48:59.793180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5151c-e5ba-4153-9b53-507160b411c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:48:59.793787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5151c-e5ba-4153-9b53-507160b411c3 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:48:59.794063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5151c-e5ba-4153-9b53-507160b411c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:48:59.794335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5151c-e5ba-4153-9b53-507160b411c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:48:59.795919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c0c5151c-e5ba-4153-9b53-507160b411c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:48:59.884880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5151c-e5ba-4153-9b53-507160b411c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:48:59.886417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5151c-e5ba-4153-9b53-507160b411c3 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:48:59.904936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 697/1393] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:00.080851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116647-89ba-4701-8a34-393f08c26cfa None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:00.081447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116647-89ba-4701-8a34-393f08c26cfa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:00.081447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116647-89ba-4701-8a34-393f08c26cfa None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:00.082759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 697/1394] 10.222.0.50 () {54 vars in 967 bytes} [Tue Aug 30 01:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:49:00.088226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27569501-4d8c-4588-ab97-08671c88ac83 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:00.088790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27569501-4d8c-4588-ab97-08671c88ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:00.088969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27569501-4d8c-4588-ab97-08671c88ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:00.166660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-27569501-4d8c-4588-ab97-08671c88ac83 None None] MFA Rules not processed for user `88273fe4807b4f08911298b7bf0c3653`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:00.167770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-27569501-4d8c-4588-ab97-08671c88ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:00.220081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:00.244033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 698/1395] 10.222.0.50 () {58 vars in 1064 bytes} [Tue Aug 30 01:49:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:00.252087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40116647-89ba-4701-8a34-393f08c26cfa None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:00.253761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-40116647-89ba-4701-8a34-393f08c26cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:00.355089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:00.362908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40116647-89ba-4701-8a34-393f08c26cfa None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:00.436506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54e5b0-f5cd-413a-b627-c9d2813f3eae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:00.437593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e5b0-f5cd-413a-b627-c9d2813f3eae service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:00.437869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e5b0-f5cd-413a-b627-c9d2813f3eae service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:00.438139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e5b0-f5cd-413a-b627-c9d2813f3eae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:00.439722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef54e5b0-f5cd-413a-b627-c9d2813f3eae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:00.546741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54e5b0-f5cd-413a-b627-c9d2813f3eae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:00.548170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54e5b0-f5cd-413a-b627-c9d2813f3eae service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:00.565801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 698/1396] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:01.292364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272de0d-abcd-4d8e-935e-4b703b783b5e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:01.292865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272de0d-abcd-4d8e-935e-4b703b783b5e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:01.293310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272de0d-abcd-4d8e-935e-4b703b783b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:01.369657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9272de0d-abcd-4d8e-935e-4b703b783b5e None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:01.370861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9272de0d-abcd-4d8e-935e-4b703b783b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:01.457386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 699/1397] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:49:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:01.498620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd53d3d-32c1-4724-8dad-4be68b9c419c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:01.499581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd53d3d-32c1-4724-8dad-4be68b9c419c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:01.499685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd53d3d-32c1-4724-8dad-4be68b9c419c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:01.501115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 699/1398] 10.222.0.50 () {54 vars in 943 bytes} [Tue Aug 30 01:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:01.507236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb12d1-8505-4073-a056-66dce856e5d9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:01.507524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb12d1-8505-4073-a056-66dce856e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:01.507823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb12d1-8505-4073-a056-66dce856e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:01.583007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6cdb12d1-8505-4073-a056-66dce856e5d9 None None] MFA Rules not processed for user `5956fdd8d6b54640970cde2705a83260`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:01.584109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6cdb12d1-8505-4073-a056-66dce856e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:01.596807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:01.693200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 700/1399] 10.222.0.50 () {58 vars in 1040 bytes} [Tue Aug 30 01:49:01 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:01.702543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd53d3d-32c1-4724-8dad-4be68b9c419c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:01.704063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4fd53d3d-32c1-4724-8dad-4be68b9c419c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:01.727296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:01.839636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd53d3d-32c1-4724-8dad-4be68b9c419c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:01.853174 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5c9ef-9b38-48fa-a63f-f93ac8018c42 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:01.853441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5c9ef-9b38-48fa-a63f-f93ac8018c42 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:01.853723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5c9ef-9b38-48fa-a63f-f93ac8018c42 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:01.855061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 701/1400] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:01.860516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:01.860797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:01.861070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:01.928456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:01.929656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:01.946975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ff97a-9172-4f13-9317-8e3194c30641 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:01.947617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ff97a-9172-4f13-9317-8e3194c30641 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:01.947907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ff97a-9172-4f13-9317-8e3194c30641 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:01.948195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ff97a-9172-4f13-9317-8e3194c30641 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:01.949869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c57ff97a-9172-4f13-9317-8e3194c30641 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:01.957215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:02.005789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:02.029259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 702/1401] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:49:02.040165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ff97a-9172-4f13-9317-8e3194c30641 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:02.041810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ff97a-9172-4f13-9317-8e3194c30641 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:02.050279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 700/1402] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 01:49:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:49:02.330307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:02.331520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:02.453940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f5c6e-323a-49de-9f11-fa9a3e2106f2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:02.552892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:02.581339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57031e3-ed69-4674-9139-09f4116404c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:02.582382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57031e3-ed69-4674-9139-09f4116404c0 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:02.582830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57031e3-ed69-4674-9139-09f4116404c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:02.583267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57031e3-ed69-4674-9139-09f4116404c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:02.586166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b57031e3-ed69-4674-9139-09f4116404c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:02.695720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57031e3-ed69-4674-9139-09f4116404c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:02.697370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57031e3-ed69-4674-9139-09f4116404c0 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:02.716558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 703/1403] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:04.657320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e5ac-9608-41a1-ab23-50b6d2877671 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:04.657857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e5ac-9608-41a1-ab23-50b6d2877671 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:04.657857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e5ac-9608-41a1-ab23-50b6d2877671 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:04.659143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 701/1404] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:04.664467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f98c-2b35-4010-b410-cf3600b09bf7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:04.664899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f98c-2b35-4010-b410-cf3600b09bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:04.665149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46f98c-2b35-4010-b410-cf3600b09bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:04.733263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7d46f98c-2b35-4010-b410-cf3600b09bf7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:04.734999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7d46f98c-2b35-4010-b410-cf3600b09bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:04.791395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:04.840536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 704/1405] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:05.141048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e5ac-9608-41a1-ab23-50b6d2877671 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:05.143087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a167e5ac-9608-41a1-ab23-50b6d2877671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:05.316591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e5ac-9608-41a1-ab23-50b6d2877671 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:05.320616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:05.472425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3acab-3f27-4304-a1c5-a14930172f7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:05.472425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3acab-3f27-4304-a1c5-a14930172f7f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:05.472425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3acab-3f27-4304-a1c5-a14930172f7f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:05.472425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3acab-3f27-4304-a1c5-a14930172f7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:05.479593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f3a3acab-3f27-4304-a1c5-a14930172f7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:05.606321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3acab-3f27-4304-a1c5-a14930172f7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:05.608316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3acab-3f27-4304-a1c5-a14930172f7f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:05.632433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 702/1406] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:07.602800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923268cc-a904-4e45-b7e4-2ebf98ce3eb0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:07.603367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923268cc-a904-4e45-b7e4-2ebf98ce3eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:07.603367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923268cc-a904-4e45-b7e4-2ebf98ce3eb0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:07.604965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 705/1407] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:07.610100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34c63-6f1a-4e19-986f-ad132c3046de None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:07.611233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34c63-6f1a-4e19-986f-ad132c3046de None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:07.611687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34c63-6f1a-4e19-986f-ad132c3046de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:07.627147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:07.698450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d8e34c63-6f1a-4e19-986f-ad132c3046de None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:07.699926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d8e34c63-6f1a-4e19-986f-ad132c3046de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:07.812619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 703/1408] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:49:08.107085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923268cc-a904-4e45-b7e4-2ebf98ce3eb0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:08.109290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-923268cc-a904-4e45-b7e4-2ebf98ce3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:08.265994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923268cc-a904-4e45-b7e4-2ebf98ce3eb0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:08.309171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:08.396292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd4554-42fe-4177-b542-5113c9666660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:08.396947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd4554-42fe-4177-b542-5113c9666660 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:08.397223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd4554-42fe-4177-b542-5113c9666660 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:08.397484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd4554-42fe-4177-b542-5113c9666660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:08.399072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b2cd4554-42fe-4177-b542-5113c9666660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:08.494583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd4554-42fe-4177-b542-5113c9666660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:08.496402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd4554-42fe-4177-b542-5113c9666660 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:08.515651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 706/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:17.810944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=62 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:18.512433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=82 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:24.043028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d061f5d-9f66-45f3-85b2-b56585bb56cf None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:24.043572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d061f5d-9f66-45f3-85b2-b56585bb56cf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:24.043572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d061f5d-9f66-45f3-85b2-b56585bb56cf None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:24.044943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 704/1410] 10.222.0.50 () {54 vars in 918 bytes} [Tue Aug 30 01:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:24.050647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f439e4-2c5f-4ebe-9cfd-4b42f1b4f294 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:24.051039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f439e4-2c5f-4ebe-9cfd-4b42f1b4f294 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:24.051188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f439e4-2c5f-4ebe-9cfd-4b42f1b4f294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:24.126473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a2f439e4-2c5f-4ebe-9cfd-4b42f1b4f294 None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:24.127660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a2f439e4-2c5f-4ebe-9cfd-4b42f1b4f294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:24.229751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 707/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Aug 30 01:49:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:24.244566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28b894-9cee-4308-97c8-b33206c4ebe5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:24.244854 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28b894-9cee-4308-97c8-b33206c4ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:24.245126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28b894-9cee-4308-97c8-b33206c4ebe5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:24.246508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 705/1412] 10.222.0.50 () {54 vars in 918 bytes} [Tue Aug 30 01:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:24.251870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cfc1a-0d47-4322-a735-058b2d093d32 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:24.252153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cfc1a-0d47-4322-a735-058b2d093d32 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:24.252461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cfc1a-0d47-4322-a735-058b2d093d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:24.323036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:24.327980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ff2cfc1a-0d47-4322-a735-058b2d093d32 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:24.329584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ff2cfc1a-0d47-4322-a735-058b2d093d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:24.442805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 708/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Aug 30 01:49:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:26.830348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c31c6-6633-4990-b348-585e4e99d902 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:26.830902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c31c6-6633-4990-b348-585e4e99d902 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:26.830902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c31c6-6633-4990-b348-585e4e99d902 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:26.832370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:26.837611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96fe28-63fc-41d4-85e9-ecb4d541362c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:26.838010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96fe28-63fc-41d4-85e9-ecb4d541362c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:26.838189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96fe28-63fc-41d4-85e9-ecb4d541362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:26.903258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1a96fe28-63fc-41d4-85e9-ecb4d541362c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:26.904468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a96fe28-63fc-41d4-85e9-ecb4d541362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:26.958436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:27.000730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:49:27.287389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c31c6-6633-4990-b348-585e4e99d902 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:27.289124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b98c31c6-6633-4990-b348-585e4e99d902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:27.430830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c31c6-6633-4990-b348-585e4e99d902 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:27.501746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:27.541231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f718723-4422-4532-a687-b8bebecc92c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:27.541763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f718723-4422-4532-a687-b8bebecc92c3 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:27.542039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f718723-4422-4532-a687-b8bebecc92c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:27.542310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f718723-4422-4532-a687-b8bebecc92c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:27.543896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0f718723-4422-4532-a687-b8bebecc92c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:27.633237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f718723-4422-4532-a687-b8bebecc92c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:27.634852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f718723-4422-4532-a687-b8bebecc92c3 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:27.653517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 707/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:28.535468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1424e-53bf-459d-8a59-3c4a11303023 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:28.536015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1424e-53bf-459d-8a59-3c4a11303023 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:28.536015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1424e-53bf-459d-8a59-3c4a11303023 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:28.537399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:49:28.542225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99189d-2ce2-48ef-b6c8-247264b1f38b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:28.542614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99189d-2ce2-48ef-b6c8-247264b1f38b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:28.542787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99189d-2ce2-48ef-b6c8-247264b1f38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:28.617860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2e99189d-2ce2-48ef-b6c8-247264b1f38b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:28.618991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e99189d-2ce2-48ef-b6c8-247264b1f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:28.645716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:28.714904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:28.866482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1424e-53bf-459d-8a59-3c4a11303023 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:28.868074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3bc1424e-53bf-459d-8a59-3c4a11303023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:29.011846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1424e-53bf-459d-8a59-3c4a11303023 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:29.050848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:29.122461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b1cbc-3401-4bbf-9e01-627f9601e568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:29.122987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1cbc-3401-4bbf-9e01-627f9601e568 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:29.123263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1cbc-3401-4bbf-9e01-627f9601e568 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:29.124037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1cbc-3401-4bbf-9e01-627f9601e568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:29.125660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-df4b1cbc-3401-4bbf-9e01-627f9601e568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:29.213796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b1cbc-3401-4bbf-9e01-627f9601e568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:29.215388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b1cbc-3401-4bbf-9e01-627f9601e568 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:29.232910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 711/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:30.114703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed0196-39bf-4c5d-8523-7d2f99e3a4b5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:30.115504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed0196-39bf-4c5d-8523-7d2f99e3a4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:30.115504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed0196-39bf-4c5d-8523-7d2f99e3a4b5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:30.116874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:30.121673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62aba7-141f-4e3f-8411-35130e28d98e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:30.122084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62aba7-141f-4e3f-8411-35130e28d98e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:30.122202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62aba7-141f-4e3f-8411-35130e28d98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:30.175592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:30.196939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dd62aba7-141f-4e3f-8411-35130e28d98e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:30.198221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dd62aba7-141f-4e3f-8411-35130e28d98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:30.295979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:30.448342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed0196-39bf-4c5d-8523-7d2f99e3a4b5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:30.449754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-83ed0196-39bf-4c5d-8523-7d2f99e3a4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:30.589658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed0196-39bf-4c5d-8523-7d2f99e3a4b5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:30.601882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:30.699797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dee674-83a6-4dcf-9255-0942356d00a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:30.700360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dee674-83a6-4dcf-9255-0942356d00a7 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:30.700641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dee674-83a6-4dcf-9255-0942356d00a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:30.700915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dee674-83a6-4dcf-9255-0942356d00a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:30.702488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b0dee674-83a6-4dcf-9255-0942356d00a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:30.788462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dee674-83a6-4dcf-9255-0942356d00a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:30.790038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dee674-83a6-4dcf-9255-0942356d00a7 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:30.807905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 710/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:31.668229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195df14c-402b-410e-ba0b-ad92ec263442 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:31.668711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195df14c-402b-410e-ba0b-ad92ec263442 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:31.668806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195df14c-402b-410e-ba0b-ad92ec263442 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:31.670136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:31.675513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714862c4-319d-4240-9d66-07c87575cbd9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:31.675974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714862c4-319d-4240-9d66-07c87575cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:31.676074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714862c4-319d-4240-9d66-07c87575cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:31.696008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:31.751593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-714862c4-319d-4240-9d66-07c87575cbd9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:31.752820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-714862c4-319d-4240-9d66-07c87575cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:31.847930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:31.998971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195df14c-402b-410e-ba0b-ad92ec263442 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:32.000528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-195df14c-402b-410e-ba0b-ad92ec263442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:32.109301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:32.145353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195df14c-402b-410e-ba0b-ad92ec263442 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:32.255087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5147ba7-c267-46aa-850d-0f0fa419baf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:32.255613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5147ba7-c267-46aa-850d-0f0fa419baf4 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:32.255893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5147ba7-c267-46aa-850d-0f0fa419baf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:32.256165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5147ba7-c267-46aa-850d-0f0fa419baf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:32.257781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d5147ba7-c267-46aa-850d-0f0fa419baf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:32.333648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:32.349669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5147ba7-c267-46aa-850d-0f0fa419baf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:32.352401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5147ba7-c267-46aa-850d-0f0fa419baf4 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:32.371090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 714/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:33.221007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11277bd-93de-4565-8469-23868b2c9332 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:33.221552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11277bd-93de-4565-8469-23868b2c9332 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:33.221552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11277bd-93de-4565-8469-23868b2c9332 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:33.222929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:33.227732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0510d-0943-4e41-9855-2d30d70333c8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:33.228137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0510d-0943-4e41-9855-2d30d70333c8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:33.228303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0510d-0943-4e41-9855-2d30d70333c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:33.303208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-aba0510d-0943-4e41-9855-2d30d70333c8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:33.304458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aba0510d-0943-4e41-9855-2d30d70333c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:33.397393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:33.545848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11277bd-93de-4565-8469-23868b2c9332 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:33.547396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f11277bd-93de-4565-8469-23868b2c9332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:33.629733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:33.691118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11277bd-93de-4565-8469-23868b2c9332 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:33.804336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0a926-59d2-4999-8eec-04b3a88c9039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:33.804855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0a926-59d2-4999-8eec-04b3a88c9039 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:33.805130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0a926-59d2-4999-8eec-04b3a88c9039 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:33.805399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0a926-59d2-4999-8eec-04b3a88c9039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:33.806976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3cf0a926-59d2-4999-8eec-04b3a88c9039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:33.853140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:33.893441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0a926-59d2-4999-8eec-04b3a88c9039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:33.895009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0a926-59d2-4999-8eec-04b3a88c9039 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:33.912234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 713/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:34.714715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178b1e2-9305-4616-95d9-0c6d1619800e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:34.715236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178b1e2-9305-4616-95d9-0c6d1619800e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:34.715236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178b1e2-9305-4616-95d9-0c6d1619800e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:34.716597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:34.722009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563bc2e-2f14-499a-b305-19cb477c8ece None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:34.722413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563bc2e-2f14-499a-b305-19cb477c8ece None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:34.722563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563bc2e-2f14-499a-b305-19cb477c8ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:34.796516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3563bc2e-2f14-499a-b305-19cb477c8ece None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:34.797718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3563bc2e-2f14-499a-b305-19cb477c8ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:34.893454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:35.044628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178b1e2-9305-4616-95d9-0c6d1619800e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:35.046149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6178b1e2-9305-4616-95d9-0c6d1619800e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:35.093653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:35.191896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178b1e2-9305-4616-95d9-0c6d1619800e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:35.301104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ee76a-2c85-45a3-88de-7d46de929c8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:35.301708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ee76a-2c85-45a3-88de-7d46de929c8a service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:35.301981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ee76a-2c85-45a3-88de-7d46de929c8a service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:35.302250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ee76a-2c85-45a3-88de-7d46de929c8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:35.303816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fa1ee76a-2c85-45a3-88de-7d46de929c8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:35.310657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:35.393313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ee76a-2c85-45a3-88de-7d46de929c8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:35.394908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ee76a-2c85-45a3-88de-7d46de929c8a service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:35.412345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 717/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:36.219154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d171c-6730-4834-bec0-643db1ff367d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:36.219702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d171c-6730-4834-bec0-643db1ff367d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:36.219702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d171c-6730-4834-bec0-643db1ff367d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:36.221082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:36.225976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6049a-bac6-4905-b124-1bbd3478f2a1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:36.226370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6049a-bac6-4905-b124-1bbd3478f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:36.226532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6049a-bac6-4905-b124-1bbd3478f2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:36.302104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-48f6049a-bac6-4905-b124-1bbd3478f2a1 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:36.303289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-48f6049a-bac6-4905-b124-1bbd3478f2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:36.368574 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:36.396828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:36.543541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d171c-6730-4834-bec0-643db1ff367d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:36.545125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a19d171c-6730-4834-bec0-643db1ff367d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:36.562812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:36.692755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d171c-6730-4834-bec0-643db1ff367d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:36.781506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:36.807131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b585-dd7e-4c1a-a8b6-dfe574578d70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:36.807666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b585-dd7e-4c1a-a8b6-dfe574578d70 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:36.807941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b585-dd7e-4c1a-a8b6-dfe574578d70 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:36.808209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b585-dd7e-4c1a-a8b6-dfe574578d70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:36.809804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-65e9b585-dd7e-4c1a-a8b6-dfe574578d70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:36.898170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b585-dd7e-4c1a-a8b6-dfe574578d70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:36.899801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b585-dd7e-4c1a-a8b6-dfe574578d70 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:36.917426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 716/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:37.715263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147faddc-42a8-4a50-941d-5d979a9abe63 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:37.715827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147faddc-42a8-4a50-941d-5d979a9abe63 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:37.715827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147faddc-42a8-4a50-941d-5d979a9abe63 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:37.717827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:37.722907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d713c-2901-49e4-8e17-2424528f268d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:37.723346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d713c-2901-49e4-8e17-2424528f268d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:37.723474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d713c-2901-49e4-8e17-2424528f268d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:37.801407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cc2d713c-2901-49e4-8e17-2424528f268d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:37.802605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cc2d713c-2901-49e4-8e17-2424528f268d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:37.843088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:37.899369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:49:38.056607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147faddc-42a8-4a50-941d-5d979a9abe63 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:38.058237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-147faddc-42a8-4a50-941d-5d979a9abe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:38.203531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147faddc-42a8-4a50-941d-5d979a9abe63 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:38.256467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:38.316903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446251d-e790-4a3c-b0da-ad47d0ac21db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:38.317512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446251d-e790-4a3c-b0da-ad47d0ac21db service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:38.317790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446251d-e790-4a3c-b0da-ad47d0ac21db service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:38.318062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446251d-e790-4a3c-b0da-ad47d0ac21db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:38.319641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9446251d-e790-4a3c-b0da-ad47d0ac21db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:38.406391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446251d-e790-4a3c-b0da-ad47d0ac21db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:38.407988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446251d-e790-4a3c-b0da-ad47d0ac21db service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:38.426184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 720/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:39.225961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062e096-f973-4ebd-9bae-83435b6361db None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:39.226510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062e096-f973-4ebd-9bae-83435b6361db None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:39.226510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062e096-f973-4ebd-9bae-83435b6361db None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:39.227808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:39.232718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ae99-c567-4913-960b-8702bd95fd19 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:39.233145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ae99-c567-4913-960b-8702bd95fd19 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:39.233265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0ae99-c567-4913-960b-8702bd95fd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:39.307472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-07a0ae99-c567-4913-960b-8702bd95fd19 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:39.308689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-07a0ae99-c567-4913-960b-8702bd95fd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:39.310663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:39.403286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:39.552102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062e096-f973-4ebd-9bae-83435b6361db None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:39.553696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9062e096-f973-4ebd-9bae-83435b6361db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:39.704786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062e096-f973-4ebd-9bae-83435b6361db None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:39.730479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:39.816303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b733437-575b-4c01-808c-5bf2ee4e010f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:39.816842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b733437-575b-4c01-808c-5bf2ee4e010f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:39.817119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b733437-575b-4c01-808c-5bf2ee4e010f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:39.817390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b733437-575b-4c01-808c-5bf2ee4e010f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:39.818956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b733437-575b-4c01-808c-5bf2ee4e010f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:39.906782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b733437-575b-4c01-808c-5bf2ee4e010f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:39.908493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b733437-575b-4c01-808c-5bf2ee4e010f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:39.926779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 719/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:41.776771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be6d5e-8de0-4a53-8479-34e7128bb3e0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:41.777320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be6d5e-8de0-4a53-8479-34e7128bb3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:41.777320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be6d5e-8de0-4a53-8479-34e7128bb3e0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:41.778682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:41.784166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72773206-4db1-4c7a-b4a9-a39aff19fdef None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:41.784618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72773206-4db1-4c7a-b4a9-a39aff19fdef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:41.784784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72773206-4db1-4c7a-b4a9-a39aff19fdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:41.825143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:41.851071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-72773206-4db1-4c7a-b4a9-a39aff19fdef None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:41.852265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72773206-4db1-4c7a-b4a9-a39aff19fdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:41.950267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:42.231139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be6d5e-8de0-4a53-8479-34e7128bb3e0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:42.233219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-03be6d5e-8de0-4a53-8479-34e7128bb3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:42.334271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be6d5e-8de0-4a53-8479-34e7128bb3e0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:42.365803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:42.406745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c30b6c-f66b-4aef-b193-ea475b0f7c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:42.407349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30b6c-f66b-4aef-b193-ea475b0f7c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:42.407624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30b6c-f66b-4aef-b193-ea475b0f7c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:42.407901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30b6c-f66b-4aef-b193-ea475b0f7c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:42.409515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c5c30b6c-f66b-4aef-b193-ea475b0f7c7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:42.515614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c30b6c-f66b-4aef-b193-ea475b0f7c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:42.517068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c30b6c-f66b-4aef-b193-ea475b0f7c7c service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:42.534935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 723/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:44.585610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d50136-bced-4533-8df8-317ca7671978 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:44.586134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d50136-bced-4533-8df8-317ca7671978 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:44.586134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d50136-bced-4533-8df8-317ca7671978 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:44.587445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:49:44.593167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cec3e-5d23-48bd-990d-58091b8e3ba9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:44.593617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cec3e-5d23-48bd-990d-58091b8e3ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:44.593741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cec3e-5d23-48bd-990d-58091b8e3ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:44.658102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bf9cec3e-5d23-48bd-990d-58091b8e3ba9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:44.659293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bf9cec3e-5d23-48bd-990d-58091b8e3ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:44.672397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:44.755729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:49:45.042746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d50136-bced-4533-8df8-317ca7671978 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:45.044312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-79d50136-bced-4533-8df8-317ca7671978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:45.147007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d50136-bced-4533-8df8-317ca7671978 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:45.158815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:45.219280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cc6a6-8d1b-4c3f-9389-f18aa57c5345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:45.219879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cc6a6-8d1b-4c3f-9389-f18aa57c5345 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:45.220153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cc6a6-8d1b-4c3f-9389-f18aa57c5345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:45.220452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cc6a6-8d1b-4c3f-9389-f18aa57c5345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:45.222019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-898cc6a6-8d1b-4c3f-9389-f18aa57c5345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:45.329915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cc6a6-8d1b-4c3f-9389-f18aa57c5345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:45.331328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cc6a6-8d1b-4c3f-9389-f18aa57c5345 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:45.348807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 722/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:47.566589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6c155-b025-479f-aead-23ee8e3263b5 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:47.567172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6c155-b025-479f-aead-23ee8e3263b5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:47.567172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6c155-b025-479f-aead-23ee8e3263b5 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:47.568514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:47.573882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77e483-747d-4cba-8cbe-ec885d62cd17 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:47.574266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77e483-747d-4cba-8cbe-ec885d62cd17 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:47.574431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77e483-747d-4cba-8cbe-ec885d62cd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:47.622005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:47.640494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9f77e483-747d-4cba-8cbe-ec885d62cd17 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:47.641711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9f77e483-747d-4cba-8cbe-ec885d62cd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:47.737625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:48.018546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6c155-b025-479f-aead-23ee8e3263b5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:48.020104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-00a6c155-b025-479f-aead-23ee8e3263b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:48.121405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6c155-b025-479f-aead-23ee8e3263b5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:48.160496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:48.194162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1157bf-f635-4514-ae20-b821618a6e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:48.194771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1157bf-f635-4514-ae20-b821618a6e55 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:48.195057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1157bf-f635-4514-ae20-b821618a6e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:48.195332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1157bf-f635-4514-ae20-b821618a6e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:48.196953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ad1157bf-f635-4514-ae20-b821618a6e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:48.303428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1157bf-f635-4514-ae20-b821618a6e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:48.304914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1157bf-f635-4514-ae20-b821618a6e55 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:48.322543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 726/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:50.453414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb0a-82f0-4be3-aa06-61a0cbe2b40c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:50.453988 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb0a-82f0-4be3-aa06-61a0cbe2b40c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:50.453988 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb0a-82f0-4be3-aa06-61a0cbe2b40c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:50.455345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:50.460532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04f48-727b-4428-8dc4-6d9fdc1cb053 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:50.460959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04f48-727b-4428-8dc4-6d9fdc1cb053 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:50.461086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04f48-727b-4428-8dc4-6d9fdc1cb053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:50.526526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-81a04f48-727b-4428-8dc4-6d9fdc1cb053 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:50.527721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-81a04f48-727b-4428-8dc4-6d9fdc1cb053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:50.548316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:50.626006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:50.914228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ffb0a-82f0-4be3-aa06-61a0cbe2b40c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:50.915806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ff6ffb0a-82f0-4be3-aa06-61a0cbe2b40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:51.017930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ffb0a-82f0-4be3-aa06-61a0cbe2b40c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:51.037190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:51.089996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49eb27-11cb-409a-bab4-79b6a3289639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:51.090596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49eb27-11cb-409a-bab4-79b6a3289639 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:51.090869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49eb27-11cb-409a-bab4-79b6a3289639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:51.091136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49eb27-11cb-409a-bab4-79b6a3289639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:51.092752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7b49eb27-11cb-409a-bab4-79b6a3289639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:51.197981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49eb27-11cb-409a-bab4-79b6a3289639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:51.199320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49eb27-11cb-409a-bab4-79b6a3289639 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:51.216887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 725/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:53.170945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423e28-fdb7-42f5-8c86-e5d9ef0d891a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:53.171522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423e28-fdb7-42f5-8c86-e5d9ef0d891a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:53.171522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423e28-fdb7-42f5-8c86-e5d9ef0d891a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:53.172877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:53.178094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976b26-d9fd-4f1b-9261-104a306c09dc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:53.178499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976b26-d9fd-4f1b-9261-104a306c09dc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:53.178645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976b26-d9fd-4f1b-9261-104a306c09dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:53.250100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:53.254126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-64976b26-d9fd-4f1b-9261-104a306c09dc None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:53.256076 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-64976b26-d9fd-4f1b-9261-104a306c09dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:53.356894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:53.640582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40423e28-fdb7-42f5-8c86-e5d9ef0d891a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:53.642235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40423e28-fdb7-42f5-8c86-e5d9ef0d891a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:53.743192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40423e28-fdb7-42f5-8c86-e5d9ef0d891a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:53.792232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:53.816689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf552b6f-80e6-4c42-8873-50a54244e8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:53.817291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf552b6f-80e6-4c42-8873-50a54244e8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:53.817565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf552b6f-80e6-4c42-8873-50a54244e8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:53.817833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf552b6f-80e6-4c42-8873-50a54244e8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:53.819405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cf552b6f-80e6-4c42-8873-50a54244e8a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:53.924331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf552b6f-80e6-4c42-8873-50a54244e8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:53.925717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf552b6f-80e6-4c42-8873-50a54244e8a5 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:53.943549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 729/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:55.958476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b4203-3e0d-430a-bb4b-5bd893932a74 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:55.959017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b4203-3e0d-430a-bb4b-5bd893932a74 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:55.959017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b4203-3e0d-430a-bb4b-5bd893932a74 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:55.960390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:55.965610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993e74-23ba-4573-8f03-10a3c00241d7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:55.966074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993e74-23ba-4573-8f03-10a3c00241d7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:55.966178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993e74-23ba-4573-8f03-10a3c00241d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:56.031126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f3993e74-23ba-4573-8f03-10a3c00241d7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:56.032383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f3993e74-23ba-4573-8f03-10a3c00241d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:56.061824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:56.129425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:56.413494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b4203-3e0d-430a-bb4b-5bd893932a74 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:56.415050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f15b4203-3e0d-430a-bb4b-5bd893932a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:56.517156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b4203-3e0d-430a-bb4b-5bd893932a74 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:56.544020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:56.588314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3a344-595e-49f6-ad93-dfe29f24090a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:56.588867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a344-595e-49f6-ad93-dfe29f24090a service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:56.589147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a344-595e-49f6-ad93-dfe29f24090a service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:56.589419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a344-595e-49f6-ad93-dfe29f24090a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:56.590995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2f3a344-595e-49f6-ad93-dfe29f24090a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:56.697694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3a344-595e-49f6-ad93-dfe29f24090a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:56.699118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3a344-595e-49f6-ad93-dfe29f24090a service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:56.716594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 728/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:49:58.692523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f1238-0005-4450-92da-f9f05a9ecc2e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:58.693058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f1238-0005-4450-92da-f9f05a9ecc2e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:58.693058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f1238-0005-4450-92da-f9f05a9ecc2e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:58.694328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:49:58.699636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402198-b72e-4e8a-a3ff-d47eb42bee39 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:58.700026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402198-b72e-4e8a-a3ff-d47eb42bee39 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:58.700187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402198-b72e-4e8a-a3ff-d47eb42bee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:58.765471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-34402198-b72e-4e8a-a3ff-d47eb42bee39 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:49:58.766657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-34402198-b72e-4e8a-a3ff-d47eb42bee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:58.774816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:58.861321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:49:59.140340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f1238-0005-4450-92da-f9f05a9ecc2e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:49:59.141905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ef1f1238-0005-4450-92da-f9f05a9ecc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:59.242685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f1238-0005-4450-92da-f9f05a9ecc2e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:49:59.297879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:49:59.316317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044652bb-ef3b-4137-b9bd-c181fd251119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:49:59.316914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044652bb-ef3b-4137-b9bd-c181fd251119 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:49:59.317187 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044652bb-ef3b-4137-b9bd-c181fd251119 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:49:59.317454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044652bb-ef3b-4137-b9bd-c181fd251119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:49:59.319005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-044652bb-ef3b-4137-b9bd-c181fd251119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:49:59.425018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044652bb-ef3b-4137-b9bd-c181fd251119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:49:59.426373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044652bb-ef3b-4137-b9bd-c181fd251119 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:49:59.445250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 732/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:01.377323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed3eff-1c1b-4055-9025-d5fe860a6207 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:01.377882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed3eff-1c1b-4055-9025-d5fe860a6207 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:01.377882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed3eff-1c1b-4055-9025-d5fe860a6207 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:01.379198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:01.384410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26240c-8e8e-4254-8a82-6495bd4944d8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:01.384845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26240c-8e8e-4254-8a82-6495bd4944d8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:01.384959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26240c-8e8e-4254-8a82-6495bd4944d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:01.451117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1d26240c-8e8e-4254-8a82-6495bd4944d8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:01.452937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1d26240c-8e8e-4254-8a82-6495bd4944d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:01.489713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:01.547831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:50:01.833183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed3eff-1c1b-4055-9025-d5fe860a6207 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:01.834721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6bed3eff-1c1b-4055-9025-d5fe860a6207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:01.937450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed3eff-1c1b-4055-9025-d5fe860a6207 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:01.974208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:02.009470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0a529-524e-4d81-8017-cd6c2fe1eced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:02.010069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a529-524e-4d81-8017-cd6c2fe1eced service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:02.010344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a529-524e-4d81-8017-cd6c2fe1eced service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:02.010615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a529-524e-4d81-8017-cd6c2fe1eced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:02.012180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c4b0a529-524e-4d81-8017-cd6c2fe1eced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:02.117625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0a529-524e-4d81-8017-cd6c2fe1eced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:02.118955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0a529-524e-4d81-8017-cd6c2fe1eced service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:02.136399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 731/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:04.038897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a7363-e8fa-4376-9fd8-e6084d1c85fe None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:04.039480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a7363-e8fa-4376-9fd8-e6084d1c85fe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:04.039480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a7363-e8fa-4376-9fd8-e6084d1c85fe None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:04.040798 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:04.045924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0def2a-236d-42a2-9e8c-2cd942892667 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:04.046352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0def2a-236d-42a2-9e8c-2cd942892667 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:04.046550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0def2a-236d-42a2-9e8c-2cd942892667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:04.114058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4b0def2a-236d-42a2-9e8c-2cd942892667 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:04.115308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b0def2a-236d-42a2-9e8c-2cd942892667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:04.133490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:04.212192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:04.497483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a7363-e8fa-4376-9fd8-e6084d1c85fe None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:04.499049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9a8a7363-e8fa-4376-9fd8-e6084d1c85fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:04.600403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a7363-e8fa-4376-9fd8-e6084d1c85fe None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:04.665443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:04.676398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa2725-5cef-4070-9784-410d1b61fef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:04.677317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa2725-5cef-4070-9784-410d1b61fef3 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:04.677746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa2725-5cef-4070-9784-410d1b61fef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:04.678216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa2725-5cef-4070-9784-410d1b61fef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:04.679834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-71aa2725-5cef-4070-9784-410d1b61fef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:04.785886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa2725-5cef-4070-9784-410d1b61fef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:04.787321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa2725-5cef-4070-9784-410d1b61fef3 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:04.804837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 735/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:06.706193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9b0f0-6489-4c4b-8176-0b63638ccd1c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:06.706736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9b0f0-6489-4c4b-8176-0b63638ccd1c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:06.706736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9b0f0-6489-4c4b-8176-0b63638ccd1c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:06.708024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:06.713266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41493a3e-0959-4159-9e54-82320563005a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:06.713694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41493a3e-0959-4159-9e54-82320563005a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:06.713806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41493a3e-0959-4159-9e54-82320563005a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:06.783209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-41493a3e-0959-4159-9e54-82320563005a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:06.784492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-41493a3e-0959-4159-9e54-82320563005a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:06.832959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:06.882130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:07.162231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9b0f0-6489-4c4b-8176-0b63638ccd1c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:07.163791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7dc9b0f0-6489-4c4b-8176-0b63638ccd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:07.265582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9b0f0-6489-4c4b-8176-0b63638ccd1c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:07.312949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:07.338252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9909e8-bb9b-41d9-936d-9c1cdec8cd28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:07.338852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9909e8-bb9b-41d9-936d-9c1cdec8cd28 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:07.339122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9909e8-bb9b-41d9-936d-9c1cdec8cd28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:07.339388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9909e8-bb9b-41d9-936d-9c1cdec8cd28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:07.340996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e9909e8-bb9b-41d9-936d-9c1cdec8cd28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:07.450365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9909e8-bb9b-41d9-936d-9c1cdec8cd28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:07.451708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9909e8-bb9b-41d9-936d-9c1cdec8cd28 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:07.469880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 734/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:16.880349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=22 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:17.467108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=66 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:26.806364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957b98-5da6-49a1-b8b9-d9489ea0a083 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:26.806921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957b98-5da6-49a1-b8b9-d9489ea0a083 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:26.806921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957b98-5da6-49a1-b8b9-d9489ea0a083 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:26.808259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 737/1471] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:26.813622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c3ef4-2fb0-4466-9b25-863dc7eea652 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:26.814022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c3ef4-2fb0-4466-9b25-863dc7eea652 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:26.814182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c3ef4-2fb0-4466-9b25-863dc7eea652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:26.879517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7b9c3ef4-2fb0-4466-9b25-863dc7eea652 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:26.880740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7b9c3ef4-2fb0-4466-9b25-863dc7eea652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:26.979464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 735/1472] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:27.281149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91d92-f3fd-47df-86e6-12ba258ff3cf None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:27.281439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91d92-f3fd-47df-86e6-12ba258ff3cf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:27.281713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91d92-f3fd-47df-86e6-12ba258ff3cf None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:27.283040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 738/1473] 10.222.0.50 () {54 vars in 967 bytes} [Tue Aug 30 01:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:27.287876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f35b90-aea5-4f2b-9238-229f5cb2d69a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:27.288163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f35b90-aea5-4f2b-9238-229f5cb2d69a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:27.288463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f35b90-aea5-4f2b-9238-229f5cb2d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:27.362755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e2f35b90-aea5-4f2b-9238-229f5cb2d69a None None] MFA Rules not processed for user `88273fe4807b4f08911298b7bf0c3653`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:27.363928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2f35b90-aea5-4f2b-9238-229f5cb2d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:27.383729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:27.440061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 736/1474] 10.222.0.50 () {58 vars in 1064 bytes} [Tue Aug 30 01:50:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:27.447521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d91d92-f3fd-47df-86e6-12ba258ff3cf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:27.449063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b9d91d92-f3fd-47df-86e6-12ba258ff3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:27.550019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d91d92-f3fd-47df-86e6-12ba258ff3cf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:27.620981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b312efb-171e-4624-9b89-292d155359d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:27.621495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312efb-171e-4624-9b89-292d155359d6 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:27.621769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312efb-171e-4624-9b89-292d155359d6 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:27.622037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312efb-171e-4624-9b89-292d155359d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:27.623584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5b312efb-171e-4624-9b89-292d155359d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:27.675541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:27.730610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b312efb-171e-4624-9b89-292d155359d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:27.732053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b312efb-171e-4624-9b89-292d155359d6 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:27.750652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 739/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:29.737131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bed2c-9577-436e-a51d-4d18db7d7259 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:29.737700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bed2c-9577-436e-a51d-4d18db7d7259 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:29.737700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bed2c-9577-436e-a51d-4d18db7d7259 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:29.739031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 737/1476] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:29.744359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345cbe0-bad5-434d-9928-8ce9fb448716 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:29.744824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345cbe0-bad5-434d-9928-8ce9fb448716 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:29.744929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345cbe0-bad5-434d-9928-8ce9fb448716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:29.808780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a345cbe0-bad5-434d-9928-8ce9fb448716 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:29.809974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a345cbe0-bad5-434d-9928-8ce9fb448716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:29.904306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 740/1477] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:30.174090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bed2c-9577-436e-a51d-4d18db7d7259 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:30.175610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-275bed2c-9577-436e-a51d-4d18db7d7259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:30.299755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bed2c-9577-436e-a51d-4d18db7d7259 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:30.348280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:30.395294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765d428-e151-455a-9537-c6a152b16d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:30.395898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765d428-e151-455a-9537-c6a152b16d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:30.396179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765d428-e151-455a-9537-c6a152b16d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:30.396482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765d428-e151-455a-9537-c6a152b16d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:30.397113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765d428-e151-455a-9537-c6a152b16d21 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:30.398872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-0765d428-e151-455a-9537-c6a152b16d21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:50:30.398872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:50:30.398872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:50:30.400044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765d428-e151-455a-9537-c6a152b16d21 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:30.418939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 738/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Tue Aug 30 01:50:30 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:50:32.319334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2632b-dfc9-49bc-89ca-12e559290b6d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:32.319809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2632b-dfc9-49bc-89ca-12e559290b6d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:32.319913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2632b-dfc9-49bc-89ca-12e559290b6d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:32.321290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 741/1479] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:32.326600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955abb-0a1e-4700-8ca6-ee2a77bc8d62 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:32.327039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955abb-0a1e-4700-8ca6-ee2a77bc8d62 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:32.327189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955abb-0a1e-4700-8ca6-ee2a77bc8d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:32.513755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a3955abb-0a1e-4700-8ca6-ee2a77bc8d62 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:32.515001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a3955abb-0a1e-4700-8ca6-ee2a77bc8d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:32.606672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 739/1480] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:32.898793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2632b-dfc9-49bc-89ca-12e559290b6d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:32.900389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-82e2632b-dfc9-49bc-89ca-12e559290b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:32.910044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:33.043710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2632b-dfc9-49bc-89ca-12e559290b6d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:33.121938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:33.154561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5b054-a5a5-41d2-a4f2-b20488a4b5aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:33.155092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b054-a5a5-41d2-a4f2-b20488a4b5aa service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:33.155368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b054-a5a5-41d2-a4f2-b20488a4b5aa service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:33.155639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5b054-a5a5-41d2-a4f2-b20488a4b5aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:33.157290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8fa5b054-a5a5-41d2-a4f2-b20488a4b5aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:33.243873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5b054-a5a5-41d2-a4f2-b20488a4b5aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:33.245499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5b054-a5a5-41d2-a4f2-b20488a4b5aa service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:33.266749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 742/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:35.329771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714dc89-b5dc-49b0-8547-07642b146097 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:35.330202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714dc89-b5dc-49b0-8547-07642b146097 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:35.330331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714dc89-b5dc-49b0-8547-07642b146097 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:35.331646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 740/1482] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:35.336981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a8178-79a5-4bd9-8f19-b1b81e521298 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:35.337387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a8178-79a5-4bd9-8f19-b1b81e521298 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:35.337523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a8178-79a5-4bd9-8f19-b1b81e521298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:35.402923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7f5a8178-79a5-4bd9-8f19-b1b81e521298 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:35.404115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7f5a8178-79a5-4bd9-8f19-b1b81e521298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:35.449065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:35.498600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 743/1483] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:35.787010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714dc89-b5dc-49b0-8547-07642b146097 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:35.788569 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0714dc89-b5dc-49b0-8547-07642b146097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:35.811371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:35.896670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714dc89-b5dc-49b0-8547-07642b146097 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:35.975594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6d230-fc6a-41e2-a102-d3bf5264d42b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:35.976109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d230-fc6a-41e2-a102-d3bf5264d42b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:35.976416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d230-fc6a-41e2-a102-d3bf5264d42b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:35.976692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6d230-fc6a-41e2-a102-d3bf5264d42b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:35.978270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bbe6d230-fc6a-41e2-a102-d3bf5264d42b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:36.051928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:36.094360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d230-fc6a-41e2-a102-d3bf5264d42b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:36.095835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6d230-fc6a-41e2-a102-d3bf5264d42b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:36.114735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 741/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:38.075440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228779-fa3d-4a63-bf83-58f43896a93f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:38.075977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228779-fa3d-4a63-bf83-58f43896a93f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:38.075977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228779-fa3d-4a63-bf83-58f43896a93f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:38.077347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 744/1485] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:38.083369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d80b-b1fc-420b-8826-3f3d3e57db9e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:38.083776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d80b-b1fc-420b-8826-3f3d3e57db9e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:38.083966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62d80b-b1fc-420b-8826-3f3d3e57db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:38.157357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1a62d80b-b1fc-420b-8826-3f3d3e57db9e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:38.158557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1a62d80b-b1fc-420b-8826-3f3d3e57db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:38.254885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 742/1486] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:38.559606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02228779-fa3d-4a63-bf83-58f43896a93f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:38.561194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-02228779-fa3d-4a63-bf83-58f43896a93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:38.662678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02228779-fa3d-4a63-bf83-58f43896a93f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:38.732505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:38.734416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb060ce-c0e5-4082-9759-41f77765833d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:38.735041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb060ce-c0e5-4082-9759-41f77765833d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:38.735510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb060ce-c0e5-4082-9759-41f77765833d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:38.735835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb060ce-c0e5-4082-9759-41f77765833d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:38.737796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fbb060ce-c0e5-4082-9759-41f77765833d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:38.965602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb060ce-c0e5-4082-9759-41f77765833d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:38.967037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb060ce-c0e5-4082-9759-41f77765833d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:38.985232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 745/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:41.074903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb308e66-ef1d-4cee-88ff-39862d2998d2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:41.075487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb308e66-ef1d-4cee-88ff-39862d2998d2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:41.075487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb308e66-ef1d-4cee-88ff-39862d2998d2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:41.076805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 743/1488] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:41.082585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eedc8-504e-45b9-b01c-a9f77ac317ad None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:41.083034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eedc8-504e-45b9-b01c-a9f77ac317ad None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:41.083145 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eedc8-504e-45b9-b01c-a9f77ac317ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:41.150043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4a4eedc8-504e-45b9-b01c-a9f77ac317ad None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:41.151258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4a4eedc8-504e-45b9-b01c-a9f77ac317ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:41.205397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:41.250898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 746/1489] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:41.546923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb308e66-ef1d-4cee-88ff-39862d2998d2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:41.548497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fb308e66-ef1d-4cee-88ff-39862d2998d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:41.583348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:41.648818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb308e66-ef1d-4cee-88ff-39862d2998d2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:41.719903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f8dee-9232-4653-92da-2a3de1817fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:41.720530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8dee-9232-4653-92da-2a3de1817fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:41.720804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8dee-9232-4653-92da-2a3de1817fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:41.721069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8dee-9232-4653-92da-2a3de1817fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:41.722624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a00f8dee-9232-4653-92da-2a3de1817fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:41.800172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:41.829643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f8dee-9232-4653-92da-2a3de1817fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:41.831059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f8dee-9232-4653-92da-2a3de1817fa6 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:41.848338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 744/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:43.925526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdecf0-80eb-4885-8ba5-176ff437df95 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:43.926090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdecf0-80eb-4885-8ba5-176ff437df95 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:43.926090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdecf0-80eb-4885-8ba5-176ff437df95 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:43.927431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 747/1491] 10.222.0.50 () {54 vars in 927 bytes} [Tue Aug 30 01:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:50:43.932806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425ac94-9fd1-46cd-89ce-789af2b016df None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:43.933236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425ac94-9fd1-46cd-89ce-789af2b016df None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:43.933349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425ac94-9fd1-46cd-89ce-789af2b016df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:43.998194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d425ac94-9fd1-46cd-89ce-789af2b016df None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:50:43.999382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d425ac94-9fd1-46cd-89ce-789af2b016df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:44.094172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 745/1492] 10.222.0.50 () {58 vars in 1024 bytes} [Tue Aug 30 01:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:50:44.382290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdecf0-80eb-4885-8ba5-176ff437df95 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:50:44.383842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5cbdecf0-80eb-4885-8ba5-176ff437df95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:44.485697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdecf0-80eb-4885-8ba5-176ff437df95 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:50:44.556596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae1d2a-1fda-4422-a7c3-2c8b3dd53cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:50:44.557130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae1d2a-1fda-4422-a7c3-2c8b3dd53cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:50:44.557406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae1d2a-1fda-4422-a7c3-2c8b3dd53cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:50:44.557677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae1d2a-1fda-4422-a7c3-2c8b3dd53cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:50:44.559231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-61ae1d2a-1fda-4422-a7c3-2c8b3dd53cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:50:44.568279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:44.681992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae1d2a-1fda-4422-a7c3-2c8b3dd53cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:50:44.683416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae1d2a-1fda-4422-a7c3-2c8b3dd53cb8 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:50:44.701146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 748/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:50:54.091908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=82 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:50:54.698942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=50 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:56.507855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:56.508466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:56.508466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:56.582329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:51:56.583439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:56.689372 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 746/1494] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:51:56.697517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49ae4-f5a6-486c-8a6b-062be9e6f883 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:56.697999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49ae4-f5a6-486c-8a6b-062be9e6f883 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:56.698106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49ae4-f5a6-486c-8a6b-062be9e6f883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:56.771278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-75c49ae4-f5a6-486c-8a6b-062be9e6f883 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:51:56.772493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75c49ae4-f5a6-486c-8a6b-062be9e6f883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:56.865334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 749/1495] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:51:56.870199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:56.871376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:56.883035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad3749-3541-4fca-acf4-dc4f23f14151 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:56.942630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:56.976625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:56.977282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:56.977559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:56.977828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:56.978459 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:51:56.978756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:56.979752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:56.979752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:51:56.979752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:51:56.980653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:56.991462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 747/1496] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:51:57.000493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49ae4-f5a6-486c-8a6b-062be9e6f883 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:57.009651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49ae4-f5a6-486c-8a6b-062be9e6f883 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:57.069061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:57.101286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:57.101884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.102159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.102432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:57.103066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:57.104812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:57.104812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Aug 30 01:51:57.104812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: Aug 30 01:51:57.105993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:57.162859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 750/1497] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:51:56 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:51:57.167051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:57.168634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:57.201634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdfe65-aafc-46a7-8eda-9297b102f676 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:57.293727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:57.294330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.294605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.294873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:57.295506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:57.296535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:57.296535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:51:57.296535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:51:57.297820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:57.342112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 748/1498] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:51:57 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:51:57.347063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:57.348681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:57.408861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e0ccf-368b-44ab-a119-a1028ee98cde admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:57.488915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:57.501197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:57.501799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.502073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.502343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:57.502973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:57.503977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:57.503977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Aug 30 01:51:57.503977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: Aug 30 01:51:57.504889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:57.517697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 751/1499] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:51:57.521702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:57.523215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:57.534490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dfaf5-3121-48c7-91b4-7778407718a3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:57.558695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:57.628134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99faa24-05ce-40ea-ba68-9fcead1b074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:57.628825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99faa24-05ce-40ea-ba68-9fcead1b074d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.629098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99faa24-05ce-40ea-ba68-9fcead1b074d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.629366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99faa24-05ce-40ea-ba68-9fcead1b074d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:57.629995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99faa24-05ce-40ea-ba68-9fcead1b074d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:57.630989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-a99faa24-05ce-40ea-ba68-9fcead1b074d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:57.630989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:51:57.630989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:51:57.631859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99faa24-05ce-40ea-ba68-9fcead1b074d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:57.644534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 749/1500] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:51:57.648758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:57.650263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:57.682979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb612a5-07a5-45e9-8782-4b2a3230cb16 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:57.774518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bfc7d-0234-4c35-9ac8-1f64ce60732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:57.775161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bfc7d-0234-4c35-9ac8-1f64ce60732a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.775435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bfc7d-0234-4c35-9ac8-1f64ce60732a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.775701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bfc7d-0234-4c35-9ac8-1f64ce60732a admin admin] PATH_INFO: `/v3/projects/a9b53688c9604b83b55937b8251d8ca3/users/8d978a6716ff4b94903c6b0c218a0a08/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:57.796575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bfc7d-0234-4c35-9ac8-1f64ce60732a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b53688c9604b83b55937b8251d8ca3, user_id=8d978a6716ff4b94903c6b0c218a0a08, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:57.798099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-d62bfc7d-0234-4c35-9ac8-1f64ce60732a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:57.798099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Aug 30 01:51:57.798099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: Aug 30 01:51:57.800033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bfc7d-0234-4c35-9ac8-1f64ce60732a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:57.823703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 752/1501] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:51:57 2022] PUT /identity/v3/projects/a9b53688c9604b83b55937b8251d8ca3/users/8d978a6716ff4b94903c6b0c218a0a08/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:51:57.835715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978601-b3f0-4eb7-be0d-fe70934ca147 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.836027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978601-b3f0-4eb7-be0d-fe70934ca147 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.836337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978601-b3f0-4eb7-be0d-fe70934ca147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:57.892984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-16978601-b3f0-4eb7-be0d-fe70934ca147 None None] MFA Rules not processed for user `8d978a6716ff4b94903c6b0c218a0a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:51:57.894171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-16978601-b3f0-4eb7-be0d-fe70934ca147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:57.969875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 750/1502] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:51:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:51:57.982706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b500-e178-45e9-ab7c-2da4e6338f11 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.982991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b500-e178-45e9-ab7c-2da4e6338f11 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.983269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b500-e178-45e9-ab7c-2da4e6338f11 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:57.984816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 753/1503] 10.222.0.50 () {54 vars in 964 bytes} [Tue Aug 30 01:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:51:57.989335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31786-1127-4242-8ba7-ff81701982dd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:57.989601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31786-1127-4242-8ba7-ff81701982dd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:57.989874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31786-1127-4242-8ba7-ff81701982dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:58.020235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:58.063304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-08b31786-1127-4242-8ba7-ff81701982dd None None] MFA Rules not processed for user `b68fc66dbef44519a2212d35655bf6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:51:58.064508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-08b31786-1127-4242-8ba7-ff81701982dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:58.138436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 751/1504] 10.222.0.50 () {58 vars in 1061 bytes} [Tue Aug 30 01:51:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:51:58.146699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b500-e178-45e9-ab7c-2da4e6338f11 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:58.148198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-31c7b500-e178-45e9-ab7c-2da4e6338f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:58.197765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:58.251876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b500-e178-45e9-ab7c-2da4e6338f11 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:58.322764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:58.323360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:58.323634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:58.323904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:58.325498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:58.421333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:58.423360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:58.426776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:58.441864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 754/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:51:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:51:58.486921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359697f-2af9-4499-af2d-9bb338013a68 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:58.487211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359697f-2af9-4499-af2d-9bb338013a68 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:58.487489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359697f-2af9-4499-af2d-9bb338013a68 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:58.488784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 752/1506] 10.222.0.50 () {52 vars in 830 bytes} [Tue Aug 30 01:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:51:58.492264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:58.493496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:58.590122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f588fa-090f-4be2-8f9b-666fdb0ecaab service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:58.680908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:58.681527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:58.681799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:58.682065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] PATH_INFO: `/v3/users/8d978a6716ff4b94903c6b0c218a0a08` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:58.694470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d978a6716ff4b94903c6b0c218a0a08)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:58.695479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:58.695479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Aug 30 01:51:58.695479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: Aug 30 01:51:58.696720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:58.706111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:58.762020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d978a6716ff4b94903c6b0c218a0a08'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:58.775816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d978a6716ff4b94903c6b0c218a0a08'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:58.785299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d978a6716ff4b94903c6b0c218a0a08'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:58.785838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d978a6716ff4b94903c6b0c218a0a08'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:58.808497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d978a6716ff4b94903c6b0c218a0a08'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:58.814580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 755/1507] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:51:58 2022] DELETE /identity/v3/users/8d978a6716ff4b94903c6b0c218a0a08 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:51:58.818942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:58.819381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:58.819501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:58.876930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:51:58.878070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:58.950780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:58.963789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 753/1508] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:51:58.973907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:58.975107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:59.070962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461812a-ccda-465c-87d9-f60b80df2c97 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:59.126464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:51:59.146696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4a0bc-a814-4e4d-be02-e2d1a4830935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:59.147238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4a0bc-a814-4e4d-be02-e2d1a4830935 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:59.147512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4a0bc-a814-4e4d-be02-e2d1a4830935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:59.147778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4a0bc-a814-4e4d-be02-e2d1a4830935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:59.149386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c7d4a0bc-a814-4e4d-be02-e2d1a4830935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:59.253926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4a0bc-a814-4e4d-be02-e2d1a4830935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:59.255347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4a0bc-a814-4e4d-be02-e2d1a4830935 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:59.273681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 756/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:51:59.624887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:51:59.626120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:51:59.637495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc5369-f8b5-4d1a-8633-fda13130b918 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:51:59.740656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:51:59.741283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:51:59.741558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:51:59.741825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] PATH_INFO: `/v3/projects/a9b53688c9604b83b55937b8251d8ca3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:51:59.752186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b53688c9604b83b55937b8251d8ca3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:51:59.753215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:51:59.753215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:51:59.753215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:51:59.754414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:51:59.810534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b53688c9604b83b55937b8251d8ca3', 'request_id': 'req-79a00324-b85b-476a-81a8-d8632b86de11'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:59.820004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b53688c9604b83b55937b8251d8ca3', 'request_id': 'req-79a00324-b85b-476a-81a8-d8632b86de11'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:59.826229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b53688c9604b83b55937b8251d8ca3', 'request_id': 'req-79a00324-b85b-476a-81a8-d8632b86de11'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:59.855519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] The token cache is being invalidate because project a9b53688c9604b83b55937b8251d8ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:51:59.855809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:59.856261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79a00324-b85b-476a-81a8-d8632b86de11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:51:59.857656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 754/1510] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:51:59 2022] DELETE /identity/v3/projects/a9b53688c9604b83b55937b8251d8ca3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:52:09.271450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=62 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:52:09.853595 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:53:28.103496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155e98c-f2d2-4773-991a-fc61a33fe7af None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:28.104054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155e98c-f2d2-4773-991a-fc61a33fe7af None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:28.104054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155e98c-f2d2-4773-991a-fc61a33fe7af None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:28.105417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:28.111184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1d294-7d00-4406-ad3d-a0ecb88414f5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:28.111639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1d294-7d00-4406-ad3d-a0ecb88414f5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:28.111755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1d294-7d00-4406-ad3d-a0ecb88414f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:28.191113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d6d1d294-7d00-4406-ad3d-a0ecb88414f5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:28.192361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6d1d294-7d00-4406-ad3d-a0ecb88414f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:28.292724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:28.447226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155e98c-f2d2-4773-991a-fc61a33fe7af None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:28.449341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a155e98c-f2d2-4773-991a-fc61a33fe7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:28.565325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155e98c-f2d2-4773-991a-fc61a33fe7af None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:28.570823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b5a8b-fac3-4fe0-8383-f79fc35ab7f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:28.571427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b5a8b-fac3-4fe0-8383-f79fc35ab7f0 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:28.571706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b5a8b-fac3-4fe0-8383-f79fc35ab7f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:28.571979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b5a8b-fac3-4fe0-8383-f79fc35ab7f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:28.580523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b5a8b-fac3-4fe0-8383-f79fc35ab7f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:28.582104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b5a8b-fac3-4fe0-8383-f79fc35ab7f0 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:28.591805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 758/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:53:29.426868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbeb8-edd7-412e-b38a-8f50f8c5008a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:29.427422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbeb8-edd7-412e-b38a-8f50f8c5008a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:29.427422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbeb8-edd7-412e-b38a-8f50f8c5008a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:29.428788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:29.433715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829d830-eba7-46a3-b02d-b86bb4eedd91 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:29.434714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829d830-eba7-46a3-b02d-b86bb4eedd91 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:29.434878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829d830-eba7-46a3-b02d-b86bb4eedd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:29.454296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8829d830-eba7-46a3-b02d-b86bb4eedd91 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:29.455464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8829d830-eba7-46a3-b02d-b86bb4eedd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:29.463173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:29.612789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbeb8-edd7-412e-b38a-8f50f8c5008a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:29.627531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbeb8-edd7-412e-b38a-8f50f8c5008a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:29.632986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753481a9-0e26-435c-8212-25a8b6a3b58a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:29.633571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753481a9-0e26-435c-8212-25a8b6a3b58a service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:29.633845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753481a9-0e26-435c-8212-25a8b6a3b58a service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:29.634115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753481a9-0e26-435c-8212-25a8b6a3b58a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:29.642183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753481a9-0e26-435c-8212-25a8b6a3b58a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:29.643740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753481a9-0e26-435c-8212-25a8b6a3b58a service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:29.652862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 757/1516] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:53:30.479757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9537e7-4b68-4b7a-9cae-963a476b3aee None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:30.480317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9537e7-4b68-4b7a-9cae-963a476b3aee None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:30.480441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9537e7-4b68-4b7a-9cae-963a476b3aee None None] PATH_INFO: `/v3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:30.481618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 760/1517] 10.222.0.57 () {54 vars in 925 bytes} [Tue Aug 30 01:53:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:53:30.488457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4baaae-6363-4062-b849-ca9bf8b8dae1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:30.488844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4baaae-6363-4062-b849-ca9bf8b8dae1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:30.489008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4baaae-6363-4062-b849-ca9bf8b8dae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:30.547078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ca4baaae-6363-4062-b849-ca9bf8b8dae1 None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:30.548187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ca4baaae-6363-4062-b849-ca9bf8b8dae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:30.564165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:53:30.617533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 758/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 30 01:53:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:39.449091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=56 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:53:39.611711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7f87-aa89-4c3f-864a-4a1965371bcd None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:39.611911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7f87-aa89-4c3f-864a-4a1965371bcd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:39.612181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7f87-aa89-4c3f-864a-4a1965371bcd None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:39.614545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 761/1519] 10.222.0.50 () {54 vars in 961 bytes} [Tue Aug 30 01:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:39.619665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76bfc-a116-42d2-8f0e-b7c7e0438422 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:39.620151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76bfc-a116-42d2-8f0e-b7c7e0438422 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:39.620275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a76bfc-a116-42d2-8f0e-b7c7e0438422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:39.640473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f2a76bfc-a116-42d2-8f0e-b7c7e0438422 None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:39.641664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f2a76bfc-a116-42d2-8f0e-b7c7e0438422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:39.650113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 759/1520] 10.222.0.50 () {58 vars in 1058 bytes} [Tue Aug 30 01:53:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:39.658164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7f87-aa89-4c3f-864a-4a1965371bcd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:39.671247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7f87-aa89-4c3f-864a-4a1965371bcd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:39.676114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1e758-42ba-43ca-86f0-f60865a58e14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:39.676729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e758-42ba-43ca-86f0-f60865a58e14 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:39.677032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e758-42ba-43ca-86f0-f60865a58e14 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:39.677308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e758-42ba-43ca-86f0-f60865a58e14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:39.685304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1e758-42ba-43ca-86f0-f60865a58e14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:39.687017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1e758-42ba-43ca-86f0-f60865a58e14 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:39.696411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 762/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:53:40.883802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644fb4-8293-45a4-8884-048f7e864568 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:40.884404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644fb4-8293-45a4-8884-048f7e864568 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:40.884404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644fb4-8293-45a4-8884-048f7e864568 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:40.885727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:40.890541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502addcb-20a2-4c01-b680-c1ab7e0808f6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:40.890980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502addcb-20a2-4c01-b680-c1ab7e0808f6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:40.891169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502addcb-20a2-4c01-b680-c1ab7e0808f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:40.910548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-502addcb-20a2-4c01-b680-c1ab7e0808f6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:40.911734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-502addcb-20a2-4c01-b680-c1ab7e0808f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:40.919453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:41.077436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30644fb4-8293-45a4-8884-048f7e864568 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:41.091663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30644fb4-8293-45a4-8884-048f7e864568 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:41.097810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcf7e4-3b0c-45e1-94f5-26963d8ce8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:41.098406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf7e4-3b0c-45e1-94f5-26963d8ce8b1 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:41.098690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf7e4-3b0c-45e1-94f5-26963d8ce8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:41.098963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf7e4-3b0c-45e1-94f5-26963d8ce8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:41.106954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcf7e4-3b0c-45e1-94f5-26963d8ce8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:41.108552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcf7e4-3b0c-45e1-94f5-26963d8ce8b1 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:41.124213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 761/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:53:41.961708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d15e1-d481-46f3-85e4-9a7fadc38d5a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:41.962584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d15e1-d481-46f3-85e4-9a7fadc38d5a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:41.963034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d15e1-d481-46f3-85e4-9a7fadc38d5a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:41.964409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:41.969383 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f4db-4e28-4162-8d1b-abb1c303d2ba None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:41.969827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f4db-4e28-4162-8d1b-abb1c303d2ba None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:41.970112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f4db-4e28-4162-8d1b-abb1c303d2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:41.990301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d155f4db-4e28-4162-8d1b-abb1c303d2ba None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:41.991495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d155f4db-4e28-4162-8d1b-abb1c303d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:42.000629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:42.158189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d15e1-d481-46f3-85e4-9a7fadc38d5a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:42.172914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d15e1-d481-46f3-85e4-9a7fadc38d5a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:42.178373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ba7c3-a5b8-4a4d-9b32-37521934c890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:42.178964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba7c3-a5b8-4a4d-9b32-37521934c890 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:42.179243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba7c3-a5b8-4a4d-9b32-37521934c890 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:42.179517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba7c3-a5b8-4a4d-9b32-37521934c890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:42.187436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ba7c3-a5b8-4a4d-9b32-37521934c890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:42.189057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ba7c3-a5b8-4a4d-9b32-37521934c890 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:42.198286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 765/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:53:45.846123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10dae1-4ac4-4bc2-aa17-261416ef193e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:45.846632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10dae1-4ac4-4bc2-aa17-261416ef193e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:45.846730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10dae1-4ac4-4bc2-aa17-261416ef193e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:45.848104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:45.853318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477e317-1455-4936-b154-32223c895918 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:45.853734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477e317-1455-4936-b154-32223c895918 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:45.853895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477e317-1455-4936-b154-32223c895918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:45.874360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0477e317-1455-4936-b154-32223c895918 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:45.875827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0477e317-1455-4936-b154-32223c895918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:45.883835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:46.169302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10dae1-4ac4-4bc2-aa17-261416ef193e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:46.171629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5f10dae1-4ac4-4bc2-aa17-261416ef193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:46.248359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10dae1-4ac4-4bc2-aa17-261416ef193e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:46.251929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70210cd-c9ea-44cf-bdf1-8f90de268f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:46.252557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70210cd-c9ea-44cf-bdf1-8f90de268f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:46.252837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70210cd-c9ea-44cf-bdf1-8f90de268f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:46.253112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70210cd-c9ea-44cf-bdf1-8f90de268f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:46.261985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70210cd-c9ea-44cf-bdf1-8f90de268f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:46.263410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70210cd-c9ea-44cf-bdf1-8f90de268f3a service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:46.273398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 764/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:53:48.149112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbd281-7067-4321-a54d-74b7beb7b7eb None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:48.149642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbd281-7067-4321-a54d-74b7beb7b7eb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:48.149642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbd281-7067-4321-a54d-74b7beb7b7eb None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:48.150915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:48.156072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ebbce-5c15-4d63-bf64-2fcdadf473e4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:48.156486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ebbce-5c15-4d63-bf64-2fcdadf473e4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:48.156655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ebbce-5c15-4d63-bf64-2fcdadf473e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:48.175650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2c9ebbce-5c15-4d63-bf64-2fcdadf473e4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:48.176842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2c9ebbce-5c15-4d63-bf64-2fcdadf473e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:48.184861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:48.474343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbd281-7067-4321-a54d-74b7beb7b7eb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:48.485084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbd281-7067-4321-a54d-74b7beb7b7eb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:48.488576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1de950-6111-499c-8442-1d6504abfc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:48.489164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1de950-6111-499c-8442-1d6504abfc50 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:48.489441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1de950-6111-499c-8442-1d6504abfc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:48.489713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1de950-6111-499c-8442-1d6504abfc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:48.498591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1de950-6111-499c-8442-1d6504abfc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:48.500014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1de950-6111-499c-8442-1d6504abfc50 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:48.509203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 768/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:53:51.834553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e30881-7dba-4e2a-939b-64bbb2f4bc24 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:51.835033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e30881-7dba-4e2a-939b-64bbb2f4bc24 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:51.835144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e30881-7dba-4e2a-939b-64bbb2f4bc24 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:51.836500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:51.841737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f96ea4-9b28-4a50-9ce5-5175092f6e6d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:51.842199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f96ea4-9b28-4a50-9ce5-5175092f6e6d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:51.842820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f96ea4-9b28-4a50-9ce5-5175092f6e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:51.862350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-65f96ea4-9b28-4a50-9ce5-5175092f6e6d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:51.863519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-65f96ea4-9b28-4a50-9ce5-5175092f6e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:51.871273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:52.423939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e30881-7dba-4e2a-939b-64bbb2f4bc24 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:52.425660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f9e30881-7dba-4e2a-939b-64bbb2f4bc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:52.489301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:53:52.502913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e30881-7dba-4e2a-939b-64bbb2f4bc24 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:52.506493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f69da1-109b-4c33-ab4d-7d140f89d068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:52.507100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f69da1-109b-4c33-ab4d-7d140f89d068 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:52.507380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f69da1-109b-4c33-ab4d-7d140f89d068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:52.507660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f69da1-109b-4c33-ab4d-7d140f89d068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:52.516868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f69da1-109b-4c33-ab4d-7d140f89d068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:52.518280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f69da1-109b-4c33-ab4d-7d140f89d068 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:52.527554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 767/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:53:54.450542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d8770-70cc-43e7-a876-5360ee27a99c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:54.451063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d8770-70cc-43e7-a876-5360ee27a99c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:54.451157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d8770-70cc-43e7-a876-5360ee27a99c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:54.453005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:53:54.458302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175ac3-766d-472b-afae-dd0e9b514fde None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:54.458727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175ac3-766d-472b-afae-dd0e9b514fde None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:54.458834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91175ac3-766d-472b-afae-dd0e9b514fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:54.478633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-91175ac3-766d-472b-afae-dd0e9b514fde None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:53:54.479828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-91175ac3-766d-472b-afae-dd0e9b514fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:54.487732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:53:55.033668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d8770-70cc-43e7-a876-5360ee27a99c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:53:55.035381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2b0d8770-70cc-43e7-a876-5360ee27a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:53:55.048548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d8770-70cc-43e7-a876-5360ee27a99c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:53:55.052064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3578e-af82-4153-ba50-9f7666791a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:53:55.052692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3578e-af82-4153-ba50-9f7666791a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:53:55.052972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3578e-af82-4153-ba50-9f7666791a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:53:55.053243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3578e-af82-4153-ba50-9f7666791a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:53:55.062497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3578e-af82-4153-ba50-9f7666791a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:53:55.063918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3578e-af82-4153-ba50-9f7666791a7b service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:53:55.073106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 771/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:54:04.473796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=11 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:54:05.069032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=33 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:17.920744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed6697-25bf-41be-af44-00cef9fa1f16 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:17.921417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed6697-25bf-41be-af44-00cef9fa1f16 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:17.921417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed6697-25bf-41be-af44-00cef9fa1f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:17.947899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f0ed6697-25bf-41be-af44-00cef9fa1f16 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:17.949315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f0ed6697-25bf-41be-af44-00cef9fa1f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:17.964064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 769/1540] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:17.973021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc90341-38ab-4d3d-adc2-e3e7dd2e84ab None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:17.973580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc90341-38ab-4d3d-adc2-e3e7dd2e84ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:17.973580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc90341-38ab-4d3d-adc2-e3e7dd2e84ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.009222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1dc90341-38ab-4d3d-adc2-e3e7dd2e84ab None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.010414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1dc90341-38ab-4d3d-adc2-e3e7dd2e84ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.026801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 772/1541] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.031290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed6697-25bf-41be-af44-00cef9fa1f16 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.046331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed6697-25bf-41be-af44-00cef9fa1f16 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.055117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.055743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.056022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.056316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.056970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:18.057268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.058919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.078842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 770/1542] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:18.088167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc90341-38ab-4d3d-adc2-e3e7dd2e84ab None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.102310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc90341-38ab-4d3d-adc2-e3e7dd2e84ab None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.108714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.109339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.109611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.109879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.110515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.117491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.170790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 773/1543] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:18.175541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.187933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5eea2-30f9-4f12-81e0-3f796256779f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.279797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.280507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.280787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.281058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.281732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.283853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.320744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 771/1544] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:18.325374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.347690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4b6ce-4ec8-4cea-a9be-e226fea59117 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.361728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.362380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.362659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.362932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.363600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.365311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.378632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 774/1545] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:18.382579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.402324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f72e-d492-4c79-8ba9-6ad51e497f96 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.413963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bb464-b1c1-4ed1-bc39-a8c3ef30b386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.414632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bb464-b1c1-4ed1-bc39-a8c3ef30b386 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.414910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bb464-b1c1-4ed1-bc39-a8c3ef30b386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.415178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bb464-b1c1-4ed1-bc39-a8c3ef30b386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.415820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bb464-b1c1-4ed1-bc39-a8c3ef30b386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.417446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bb464-b1c1-4ed1-bc39-a8c3ef30b386 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.438216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 772/1546] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:18.442391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.457101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be4cea-5312-4236-a0bb-81e71861abac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.468706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990cda7-37b0-46d2-ae25-620ab4e4f250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.469402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990cda7-37b0-46d2-ae25-620ab4e4f250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.469677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990cda7-37b0-46d2-ae25-620ab4e4f250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.469949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990cda7-37b0-46d2-ae25-620ab4e4f250 admin admin] PATH_INFO: `/v3/projects/56a61719b5c8402b9542dacad4242644/users/b74a7caa5628414bbbf9bab161625f0d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.500192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990cda7-37b0-46d2-ae25-620ab4e4f250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56a61719b5c8402b9542dacad4242644, user_id=b74a7caa5628414bbbf9bab161625f0d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.503316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990cda7-37b0-46d2-ae25-620ab4e4f250 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.525159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a9244-3f70-4f2c-867b-a7aeb7551db5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.525557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a9244-3f70-4f2c-867b-a7aeb7551db5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.525920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a9244-3f70-4f2c-867b-a7aeb7551db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.529004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 775/1547] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:18 2022] PUT /identity/v3/projects/56a61719b5c8402b9542dacad4242644/users/b74a7caa5628414bbbf9bab161625f0d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:18.564800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-542a9244-3f70-4f2c-867b-a7aeb7551db5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.566091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-542a9244-3f70-4f2c-867b-a7aeb7551db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.583230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925f2be-9ae6-4a1d-b01e-5b8b36de0f50 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.583792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925f2be-9ae6-4a1d-b01e-5b8b36de0f50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.584093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925f2be-9ae6-4a1d-b01e-5b8b36de0f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.646492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2925f2be-9ae6-4a1d-b01e-5b8b36de0f50 None None] MFA Rules not processed for user `b74a7caa5628414bbbf9bab161625f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.647607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2925f2be-9ae6-4a1d-b01e-5b8b36de0f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.671045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 773/1548] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:55:18.672348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee471be-f837-4f3f-be76-5209d7ffb50c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.672632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee471be-f837-4f3f-be76-5209d7ffb50c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.672902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee471be-f837-4f3f-be76-5209d7ffb50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.708045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6ee471be-f837-4f3f-be76-5209d7ffb50c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.709264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6ee471be-f837-4f3f-be76-5209d7ffb50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.739635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 774/1549] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.740955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac559e2-ab23-4958-8187-1806bf9e2027 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.741235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac559e2-ab23-4958-8187-1806bf9e2027 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.741505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac559e2-ab23-4958-8187-1806bf9e2027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.776334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-aac559e2-ab23-4958-8187-1806bf9e2027 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.776334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aac559e2-ab23-4958-8187-1806bf9e2027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.778809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 776/1550] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.787759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400400ef-2416-416f-8e39-25a1583f811d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.788130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400400ef-2416-416f-8e39-25a1583f811d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.788438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400400ef-2416-416f-8e39-25a1583f811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.809280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 775/1551] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.811065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0d452-242c-44a7-aafb-e8908cbfb921 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.811562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0d452-242c-44a7-aafb-e8908cbfb921 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.812061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0d452-242c-44a7-aafb-e8908cbfb921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.830401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-400400ef-2416-416f-8e39-25a1583f811d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.832429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-400400ef-2416-416f-8e39-25a1583f811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.842315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 777/1552] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.847867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-05d0d452-242c-44a7-aafb-e8908cbfb921 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.848029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85123cee-c6cc-4cab-8a37-aeeb8b0c8a7d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.848117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-05d0d452-242c-44a7-aafb-e8908cbfb921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.848666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85123cee-c6cc-4cab-8a37-aeeb8b0c8a7d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.848795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85123cee-c6cc-4cab-8a37-aeeb8b0c8a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.854662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 776/1553] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.855715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ef3b-8a4d-459d-9b7b-36adf08753c8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.855999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ef3b-8a4d-459d-9b7b-36adf08753c8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.856299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ef3b-8a4d-459d-9b7b-36adf08753c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.870300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-85123cee-c6cc-4cab-8a37-aeeb8b0c8a7d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.871476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-85123cee-c6cc-4cab-8a37-aeeb8b0c8a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.875058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5b29ef3b-8a4d-459d-9b7b-36adf08753c8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.876206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5b29ef3b-8a4d-459d-9b7b-36adf08753c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.880135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 778/1554] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.881516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686feb-b127-4dde-9b3c-aed3527c101f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.881867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686feb-b127-4dde-9b3c-aed3527c101f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.882256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94686feb-b127-4dde-9b3c-aed3527c101f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.885557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 777/1555] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.886817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e8781-528e-4c51-8301-899725f5134b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.887104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e8781-528e-4c51-8301-899725f5134b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.887377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e8781-528e-4c51-8301-899725f5134b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.888829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 778/1556] 10.222.0.50 () {54 vars in 962 bytes} [Tue Aug 30 01:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:55:18.894416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1801d5-c150-4eac-9c52-7ecefa97cfa9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.894700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1801d5-c150-4eac-9c52-7ecefa97cfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.894972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1801d5-c150-4eac-9c52-7ecefa97cfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.903397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-94686feb-b127-4dde-9b3c-aed3527c101f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.904538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-94686feb-b127-4dde-9b3c-aed3527c101f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.912907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 779/1557] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.913752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94686feb-b127-4dde-9b3c-aed3527c101f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.914324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4e1801d5-c150-4eac-9c52-7ecefa97cfa9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.915547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e1801d5-c150-4eac-9c52-7ecefa97cfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.923101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94686feb-b127-4dde-9b3c-aed3527c101f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.923509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 779/1558] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.924592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc93aff-748f-434a-83bd-c32c9ff10c1e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.924876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc93aff-748f-434a-83bd-c32c9ff10c1e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.925144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc93aff-748f-434a-83bd-c32c9ff10c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.935272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.935881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.936152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.936465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.937116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:18.937414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.938424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:55:18.938424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Aug 30 01:55:18.938424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: Aug 30 01:55:18.939331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf81ed1-3125-427b-a5cd-195403c530fc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.943757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2fc93aff-748f-434a-83bd-c32c9ff10c1e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.945011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2fc93aff-748f-434a-83bd-c32c9ff10c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.958543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 780/1559] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:18.958543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded004-16c8-4369-8249-1c5d2d11067a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.958543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded004-16c8-4369-8249-1c5d2d11067a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.958543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded004-16c8-4369-8249-1c5d2d11067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.958931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 780/1560] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.958931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc93aff-748f-434a-83bd-c32c9ff10c1e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:18.966067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc93aff-748f-434a-83bd-c32c9ff10c1e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:18.971263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc0004-cfed-4201-be73-5c0b2beb0231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:18.971856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0004-cfed-4201-be73-5c0b2beb0231 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.972129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0004-cfed-4201-be73-5c0b2beb0231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.972429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0004-cfed-4201-be73-5c0b2beb0231 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.973061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc0004-cfed-4201-be73-5c0b2beb0231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:18.973180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-aeded004-16c8-4369-8249-1c5d2d11067a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:18.973629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc0004-cfed-4201-be73-5c0b2beb0231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:18.974246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aeded004-16c8-4369-8249-1c5d2d11067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:18.974922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc0004-cfed-4201-be73-5c0b2beb0231 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:18.982137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 781/1561] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:18.985957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59413fd-068c-43a4-ad3f-0244e0f2e559 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.985957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59413fd-068c-43a4-ad3f-0244e0f2e559 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.985957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59413fd-068c-43a4-ad3f-0244e0f2e559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:18.986417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 781/1562] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:18.987634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643fa31-0638-4e83-bdc6-0e50fcb9bd95 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:18.988132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643fa31-0638-4e83-bdc6-0e50fcb9bd95 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:18.988435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643fa31-0638-4e83-bdc6-0e50fcb9bd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.003115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:19.009534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9643fa31-0638-4e83-bdc6-0e50fcb9bd95 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:19.010718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9643fa31-0638-4e83-bdc6-0e50fcb9bd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:19.019056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 782/1563] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:19.019828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643fa31-0638-4e83-bdc6-0e50fcb9bd95 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.025834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d59413fd-068c-43a4-ad3f-0244e0f2e559 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:19.027027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d59413fd-068c-43a4-ad3f-0244e0f2e559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:19.028883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643fa31-0638-4e83-bdc6-0e50fcb9bd95 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.034058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dbab9-aaa9-447a-8421-5ff2a643e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.034639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dbab9-aaa9-447a-8421-5ff2a643e316 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.034912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dbab9-aaa9-447a-8421-5ff2a643e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.035181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dbab9-aaa9-447a-8421-5ff2a643e316 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.035835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dbab9-aaa9-447a-8421-5ff2a643e316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:19.036133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dbab9-aaa9-447a-8421-5ff2a643e316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.037736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dbab9-aaa9-447a-8421-5ff2a643e316 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.046688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:19.049469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 783/1564] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:19.050756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42349b42-313d-4306-b449-60627d9154a6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.051129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42349b42-313d-4306-b449-60627d9154a6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.051416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42349b42-313d-4306-b449-60627d9154a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.070837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-42349b42-313d-4306-b449-60627d9154a6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:19.072020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-42349b42-313d-4306-b449-60627d9154a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:19.080578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 784/1565] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:19.081316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42349b42-313d-4306-b449-60627d9154a6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.090766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42349b42-313d-4306-b449-60627d9154a6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.095331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.096676 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.096961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.097236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.097853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.098844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:55:19.098844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:55:19.098844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:55:19.099482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 782/1566] 10.222.0.50 () {58 vars in 1059 bytes} [Tue Aug 30 01:55:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:19.100026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.100886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f20d-4b84-4c42-a654-5d8256d37ec9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.101176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f20d-4b84-4c42-a654-5d8256d37ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.101451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f20d-4b84-4c42-a654-5d8256d37ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.121068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-77a7f20d-4b84-4c42-a654-5d8256d37ec9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:19.122259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-77a7f20d-4b84-4c42-a654-5d8256d37ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:19.128375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 785/1567] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.128375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.133240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 783/1568] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:19.134126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7f20d-4b84-4c42-a654-5d8256d37ec9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.144367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7f20d-4b84-4c42-a654-5d8256d37ec9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.190438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fa157-baa3-4303-ae36-64cd765baa2f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.195021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.195625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.195906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.196180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.196870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:19.197172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.198657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.198783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.199255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.199529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.199797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.200444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.202321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.211116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 786/1569] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:19.212024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.221690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd9c3f-6ef2-4b97-a3e8-2e292d08563c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.226269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.226941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.227226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.227502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.228258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:19.228481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.230057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.238237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 784/1570] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.238237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.250965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 787/1571] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:19.250965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.255250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc58a7-418c-4d01-a020-99543225be27 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.296438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84187094-4a89-4601-a522-03dfd10c33a7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.301112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.301713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.301986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.302257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.302885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.304849 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.309879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.310491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.310766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.311037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.311674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:19.311977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.313584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.333379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 788/1572] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:19.333379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.338278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 785/1573] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.338278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.356463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:19.365660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:19.399415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a13566-b3d4-462e-b7d3-b09d6de54851 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.403638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd468-8c47-4fa9-972f-a1fd4e483030 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.404043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.404683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.404959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.405244 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.405867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.406878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:55:19.406878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Aug 30 01:55:19.406878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: Aug 30 01:55:19.408648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.431139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 786/1574] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.432350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.442014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cdc8e8-d203-48e2-a3cb-ecbc51c7c343 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.446697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.447287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.447601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.447875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.448552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:19.448858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.450487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.462717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 787/1575] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:19.463489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.469376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.470067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.470344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.471107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.475162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bbbc7-44b9-418b-9740-f1e9e0be4270 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.479724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.480324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.480600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.480871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.481492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.483383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.510143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 788/1576] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.511014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.530986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.532612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.542173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 789/1577] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:19.542940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.552738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94253f85-f6c9-49bc-a2b6-d13c375bd01e service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.573811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f6539-96a1-4135-b1d2-6e0f6060776e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.578543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.579269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.579556 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.580387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.581105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.583657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.610483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 789/1578] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.611259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.621268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8ab04-885d-4b0b-b933-f750d0fa56bc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.623321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.624122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.624439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.624968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.625599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.627094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.627667 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.627777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.627982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.628400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.629019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.630888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.642838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:19.661819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 790/1579] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.662031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 790/1580] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.662143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.662261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.713051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:19.734154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f847a4-cdc6-406a-9c83-77ca6ef57e09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.734479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e528c2e-667b-4b30-8ebd-754a7e41ceb8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.738862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.739355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.739493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.739746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.739985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.740113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.740332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.740570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.740688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.741319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.742567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.742902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.753637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 791/1581] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:19.754231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.765173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876603f-0841-4952-9d1e-e24cf9c02383 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.765359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 791/1582] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.769785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.770612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.771217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.771486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.771755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.772402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.774289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.779980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae8ed6-5b11-4512-b011-53e97e5b58e9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.785266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.785832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.786111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.786382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.787099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.789043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.804672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 792/1583] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.805778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.811642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 792/1584] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.812574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.822356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a71f4-76ad-4eca-a613-7a86109ea7c4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.868447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4101fa-717c-4054-af91-ce25a5f2d3fe admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.875983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.875983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.875983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.875983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.876441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.876737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.881832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.882445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.883264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.883544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.884207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.886527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.887471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 793/1585] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:19.888265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.899585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba6e9c-83a9-46c7-a619-5f63763cd7f2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.905124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.905806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.906264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.906540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.907489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.910181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.915327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 793/1586] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.916346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.926701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ecf72-7c97-4fac-b474-281d516acda9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.931394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801067b6-6f92-4301-9e30-c57c571f1b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.931989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801067b6-6f92-4301-9e30-c57c571f1b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.932358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801067b6-6f92-4301-9e30-c57c571f1b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.932662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801067b6-6f92-4301-9e30-c57c571f1b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.933290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 794/1587] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.933462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801067b6-6f92-4301-9e30-c57c571f1b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.934129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.934953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801067b6-6f92-4301-9e30-c57c571f1b00 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.946178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0f7ae-16ea-40ca-9371-fac6dfd4cd6e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.946477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 794/1588] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:19.947867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13a0a-0358-454d-b271-7ccfee18515e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.948157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13a0a-0358-454d-b271-7ccfee18515e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.948453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13a0a-0358-454d-b271-7ccfee18515e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.949758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 795/1589] 10.222.0.50 () {54 vars in 912 bytes} [Tue Aug 30 01:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:55:19.950498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13a0a-0358-454d-b271-7ccfee18515e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.957805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.958442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.958717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.959037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.959686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.961408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.964013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13a0a-0358-454d-b271-7ccfee18515e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.969118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.969701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.969975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.970377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.971036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.975029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 795/1590] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:19.975029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:19.975564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.982019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba558a3a-d91d-4815-8683-82e8fc195e58 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:19.986812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:19.987395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:19.987668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:19.987944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:19.988617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:19.990254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:19.995681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 796/1591] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:19 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:19.996540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.002279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 796/1592] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.003389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.006942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a09a83-2c83-410f-bed1-df964b378a91 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.011677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.012281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.012561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.012828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.013151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a403a61-4c90-4258-a270-f32f3fc1cb81 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.013456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.015019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.018251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.018820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.019127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.019401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.020026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.021634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.027191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 797/1593] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.027992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.031795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 797/1594] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.032591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.039734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1169f1-767a-42b4-baaa-1e24264da708 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.043175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd03eb-1273-4dbb-9dd7-4ca09e39c3ac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.044616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc4715-c151-43b9-b60f-4cb03ca0129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.045204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4715-c151-43b9-b60f-4cb03ca0129a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.045484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4715-c151-43b9-b60f-4cb03ca0129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.045764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4715-c151-43b9-b60f-4cb03ca0129a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.046418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc4715-c151-43b9-b60f-4cb03ca0129a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.047851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.047976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc4715-c151-43b9-b60f-4cb03ca0129a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.048507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.048810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.049186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] PATH_INFO: `/v3/projects/4b0db8e5fc6c48cc98136ef2a85b05f4/users/85d2f3d2a8cb40d38d0ca2dc16609ef5/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.057825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 798/1595] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.058839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c15bf9-4142-440c-9815-2b91c1d4e36d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.059128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c15bf9-4142-440c-9815-2b91c1d4e36d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.059404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c15bf9-4142-440c-9815-2b91c1d4e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.064126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0db8e5fc6c48cc98136ef2a85b05f4, user_id=85d2f3d2a8cb40d38d0ca2dc16609ef5, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.067552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.069192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=13,keystone:SELECT=87 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:20.082484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 798/1596] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:20 2022] PUT /identity/v3/projects/4b0db8e5fc6c48cc98136ef2a85b05f4/users/85d2f3d2a8cb40d38d0ca2dc16609ef5/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:20.082684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-09c15bf9-4142-440c-9815-2b91c1d4e36d None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:20.083283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.083697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-09c15bf9-4142-440c-9815-2b91c1d4e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:20.125918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:20.150097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a05f-2bd2-48f6-8517-fe3f87662768 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.151218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 799/1597] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:55:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:20.153404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c15bf9-4142-440c-9815-2b91c1d4e36d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.155707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.156451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.156731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.157133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.157779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.159421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.164520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c15bf9-4142-440c-9815-2b91c1d4e36d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.169907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.170065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 799/1598] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.170311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.170590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.170709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.170898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.171498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.173228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.182198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d171-b4b3-436d-9c96-964311ffff92 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.183466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 800/1599] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.184222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.190055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.190641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.190918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.191193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.191827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.193489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.194265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab268293-c28b-483f-ac58-42caf95836bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.199555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119afed6-9e61-4ebd-8b5b-16d3e435d404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.200220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119afed6-9e61-4ebd-8b5b-16d3e435d404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.200527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119afed6-9e61-4ebd-8b5b-16d3e435d404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.200965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119afed6-9e61-4ebd-8b5b-16d3e435d404 admin admin] PATH_INFO: `/v3/projects/abac996e90424840a7e39b2620b838d7/users/b6bd8657bf5f4dc18cc0ead8e501a176/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.207466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 800/1600] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.207466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.214812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4a1db-c07a-44da-880e-6e9e9cf7bee0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.217176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119afed6-9e61-4ebd-8b5b-16d3e435d404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abac996e90424840a7e39b2620b838d7, user_id=b6bd8657bf5f4dc18cc0ead8e501a176, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.218217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-119afed6-9e61-4ebd-8b5b-16d3e435d404 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:55:20.218217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:55:20.218217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:55:20.219642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.220155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119afed6-9e61-4ebd-8b5b-16d3e435d404 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.220296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.220563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.220833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.221475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.223073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.248273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 801/1601] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.248273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.248603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 801/1602] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:20 2022] PUT /identity/v3/projects/abac996e90424840a7e39b2620b838d7/users/b6bd8657bf5f4dc18cc0ead8e501a176/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:20.248603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07099d9a-dfc3-415c-927f-3908844c0ae5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.248603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07099d9a-dfc3-415c-927f-3908844c0ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.248603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07099d9a-dfc3-415c-927f-3908844c0ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.248953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406365a3-f480-4431-bbd1-c1de94fc432b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.259914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-07099d9a-dfc3-415c-927f-3908844c0ae5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:20.261431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-07099d9a-dfc3-415c-927f-3908844c0ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:20.309275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.309904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.310176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.310443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.311100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.312718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.321060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 802/1603] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:20.324350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07099d9a-dfc3-415c-927f-3908844c0ae5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.324441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 802/1604] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.324441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.336555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10742f6d-e2e4-4ef0-9879-b4d409b42ae7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.341406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.342047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.342325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.342597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] PATH_INFO: `/v3/projects/b9d96d936ce8417197008a59da394df7/users/a06de37941264b979a3bd5aaf84ec73c/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.357822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d96d936ce8417197008a59da394df7, user_id=a06de37941264b979a3bd5aaf84ec73c, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.360641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.373890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 803/1605] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:20 2022] PUT /identity/v3/projects/b9d96d936ce8417197008a59da394df7/users/a06de37941264b979a3bd5aaf84ec73c/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:20.375020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.383637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07099d9a-dfc3-415c-927f-3908844c0ae5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.396822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:20.434526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc491407-4237-43ff-85bf-e71b3b0cda1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.435135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc491407-4237-43ff-85bf-e71b3b0cda1c service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.435413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc491407-4237-43ff-85bf-e71b3b0cda1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.435688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc491407-4237-43ff-85bf-e71b3b0cda1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.438348 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502f456-1ed3-4ddd-bc1f-608ac0cfc733 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.443779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.444421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.444704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.444975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.445648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.447241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.456923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 804/1606] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:20.457688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.467331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d116f-92c6-4f08-ad30-d0674ab17101 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.472258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.472932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.473208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.473487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] PATH_INFO: `/v3/projects/830abb133bfb4de0a791bf110baab71e/users/5b5da237ee034b3a85285a671d30223c/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.488712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830abb133bfb4de0a791bf110baab71e, user_id=5b5da237ee034b3a85285a671d30223c, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.491389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.494587 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:20.506528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 805/1607] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:20 2022] PUT /identity/v3/projects/830abb133bfb4de0a791bf110baab71e/users/5b5da237ee034b3a85285a671d30223c/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:20.517253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.519981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc491407-4237-43ff-85bf-e71b3b0cda1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.521457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc491407-4237-43ff-85bf-e71b3b0cda1c service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.532872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 803/1608] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:20.534118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a252e1b-1130-47a8-8df7-d0595288f5ef None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.534462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a252e1b-1130-47a8-8df7-d0595288f5ef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.534844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a252e1b-1130-47a8-8df7-d0595288f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.563327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2a252e1b-1130-47a8-8df7-d0595288f5ef None None] MFA Rules not processed for user `b6bd8657bf5f4dc18cc0ead8e501a176`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:20.565240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2a252e1b-1130-47a8-8df7-d0595288f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:20.573795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18529c-946f-4b46-a87c-e057ee075c70 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.580170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.580861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.581140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.581416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] PATH_INFO: `/v3/projects/f4a7212d15b6463f834e8d1cc4ebf685/users/cc7381cfffee4842bee1950a18caf807/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.598733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a7212d15b6463f834e8d1cc4ebf685, user_id=cc7381cfffee4842bee1950a18caf807, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.601461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.620995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 806/1609] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:20 2022] PUT /identity/v3/projects/f4a7212d15b6463f834e8d1cc4ebf685/users/cc7381cfffee4842bee1950a18caf807/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:20.621584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.635115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 804/1610] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:20.638762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a252e1b-1130-47a8-8df7-d0595288f5ef None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.688490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6dbb7-66aa-453e-952f-47f05f87950c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.693844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeac046-8313-4cc7-a54c-86c801ac38d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.694576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeac046-8313-4cc7-a54c-86c801ac38d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.694918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeac046-8313-4cc7-a54c-86c801ac38d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.695251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeac046-8313-4cc7-a54c-86c801ac38d1 admin admin] PATH_INFO: `/v3/projects/4ad56c2e3e7b4d35831b3c8f72db6fcc/users/e83d4a4186a649899593fa5443425147/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.696884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:20.700748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a252e1b-1130-47a8-8df7-d0595288f5ef None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.712518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeac046-8313-4cc7-a54c-86c801ac38d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad56c2e3e7b4d35831b3c8f72db6fcc, user_id=e83d4a4186a649899593fa5443425147, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.715270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeac046-8313-4cc7-a54c-86c801ac38d1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.740609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 807/1611] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:20 2022] PUT /identity/v3/projects/4ad56c2e3e7b4d35831b3c8f72db6fcc/users/e83d4a4186a649899593fa5443425147/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:20.740609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a52d-f94e-428a-8a43-3744f67a61e6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.740609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a52d-f94e-428a-8a43-3744f67a61e6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.740609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a52d-f94e-428a-8a43-3744f67a61e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.751100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-06c8a52d-f94e-428a-8a43-3744f67a61e6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:20.752355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-06c8a52d-f94e-428a-8a43-3744f67a61e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:20.756945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.757641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.757916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.758179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.768209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.769705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.780634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 805/1612] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:20.781490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:20.792656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fcdde-aa21-4bb7-a0d4-9cf6f861d02e service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:20.827685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 808/1613] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:20.831777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4493-acaa-4914-8497-6e73fcb54554 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.831777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4493-acaa-4914-8497-6e73fcb54554 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.831777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4493-acaa-4914-8497-6e73fcb54554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.847979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:20.871056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbbef4-181f-45bc-a23d-299713775b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:20.871731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbbef4-181f-45bc-a23d-299713775b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.872009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbbef4-181f-45bc-a23d-299713775b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.872307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbbef4-181f-45bc-a23d-299713775b9e admin admin] PATH_INFO: `/v3/projects/f90f06a31df841b89ed6874d20340475/users/68fcefa4a0db4bd6a051b1446e1a0d2d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.887464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-42bc4493-acaa-4914-8497-6e73fcb54554 None None] MFA Rules not processed for user `5b5da237ee034b3a85285a671d30223c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:20.889024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-42bc4493-acaa-4914-8497-6e73fcb54554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:20.895814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbbef4-181f-45bc-a23d-299713775b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90f06a31df841b89ed6874d20340475, user_id=68fcefa4a0db4bd6a051b1446e1a0d2d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:20.900303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbbef4-181f-45bc-a23d-299713775b9e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:20.921302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 806/1614] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:20 2022] PUT /identity/v3/projects/f90f06a31df841b89ed6874d20340475/users/68fcefa4a0db4bd6a051b1446e1a0d2d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:20.923787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacae39-6e64-4db0-b3fa-478bdd463cd3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:20.924381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacae39-6e64-4db0-b3fa-478bdd463cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:20.924870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacae39-6e64-4db0-b3fa-478bdd463cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:20.955738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-feacae39-6e64-4db0-b3fa-478bdd463cd3 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:20.957023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-feacae39-6e64-4db0-b3fa-478bdd463cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:20.987045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 809/1615] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:20.988309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4493-acaa-4914-8497-6e73fcb54554 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.001233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4493-acaa-4914-8497-6e73fcb54554 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.078884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.079642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.080016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.080338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.081555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.083048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.084075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 807/1616] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:21.085404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0bba9-46d2-49d8-be4a-b51bb37489c8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.085683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0bba9-46d2-49d8-be4a-b51bb37489c8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.085953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0bba9-46d2-49d8-be4a-b51bb37489c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.130385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 810/1617] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:20 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:21.132461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.133471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8db0bba9-46d2-49d8-be4a-b51bb37489c8 None None] MFA Rules not processed for user `e83d4a4186a649899593fa5443425147`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:21.134707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8db0bba9-46d2-49d8-be4a-b51bb37489c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:21.135706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:21.156888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:21.249112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 808/1618] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:21.251330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52ffc-0de6-4c80-aea9-c7f0a3ab17eb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.252333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52ffc-0de6-4c80-aea9-c7f0a3ab17eb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.253360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52ffc-0de6-4c80-aea9-c7f0a3ab17eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.275743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c0c52ffc-0de6-4c80-aea9-c7f0a3ab17eb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:21.277517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c0c52ffc-0de6-4c80-aea9-c7f0a3ab17eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:21.279377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ef022-e556-4a20-9e98-57f939be212e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.283531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.284159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.284469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.284748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.363302 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:21.421121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.423121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.428871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 809/1619] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:21.432119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52ffc-0de6-4c80-aea9-c7f0a3ab17eb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.458581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 811/1620] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:21.460137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.468562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b989cbc-b29d-4914-a745-1a9fdf55a420 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.473303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.474050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.474334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.474609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.487709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.489260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.501549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 812/1621] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:21.502674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.511872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52ffc-0de6-4c80-aea9-c7f0a3ab17eb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.514243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baa611-3f82-4abd-b669-b27207917a5f service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.516059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869d008-51cc-4017-92e7-85a2abb4baf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.517075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869d008-51cc-4017-92e7-85a2abb4baf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.517609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869d008-51cc-4017-92e7-85a2abb4baf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.518087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869d008-51cc-4017-92e7-85a2abb4baf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.519795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.520501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.520886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.521253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.522254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.524207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.554068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 813/1622] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:21 2022] POST /identity/v3/users => generated 294 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:21.555481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.567386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1235bb-480b-41c9-96ef-167635623907 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.572975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.573695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.574162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.574532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.575312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.577389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.600631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869d008-51cc-4017-92e7-85a2abb4baf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.602218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869d008-51cc-4017-92e7-85a2abb4baf8 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.611093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 814/1623] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:21 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:21.612137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.612584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 810/1624] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:21.614018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3ef3-3474-4b7f-a298-10adbdac6bbb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.614461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3ef3-3474-4b7f-a298-10adbdac6bbb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.614841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3ef3-3474-4b7f-a298-10adbdac6bbb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.616533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 811/1625] 10.222.0.50 () {54 vars in 962 bytes} [Tue Aug 30 01:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:55:21.617310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb3ef3-3474-4b7f-a298-10adbdac6bbb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.641315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:21.694630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc1e56-c5c9-4687-9e4f-a4bb2e5ecad4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.696711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb3ef3-3474-4b7f-a298-10adbdac6bbb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.699576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aacc76-fe45-4838-8331-9dd4338cbf05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.700664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aacc76-fe45-4838-8331-9dd4338cbf05 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.700964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aacc76-fe45-4838-8331-9dd4338cbf05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.701241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aacc76-fe45-4838-8331-9dd4338cbf05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.704225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.705177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.706412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.706883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.707850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.709940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.713275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aacc76-fe45-4838-8331-9dd4338cbf05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.714727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aacc76-fe45-4838-8331-9dd4338cbf05 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.724464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 812/1626] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:21.725777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.726935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 815/1627] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:21.728502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dc992-55ab-4db3-a20a-f543ac3ffdf3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.729009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dc992-55ab-4db3-a20a-f543ac3ffdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.729287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dc992-55ab-4db3-a20a-f543ac3ffdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.745143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcab9a-335b-433a-82b7-a846ffb84b3a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.751904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.752882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.753228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.753538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-024dc992-55ab-4db3-a20a-f543ac3ffdf3 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:21.753712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.754577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.755345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-024dc992-55ab-4db3-a20a-f543ac3ffdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:21.757490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.790012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 813/1628] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:21 2022] POST /identity/v3/users => generated 297 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:21.791321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.804838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b712249-ae55-4365-9b4e-36445674ed38 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.811033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.811770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.812482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.812768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] PATH_INFO: `/v3/users/5b5da237ee034b3a85285a671d30223c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.814094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5da237ee034b3a85285a671d30223c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.815537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:55:21.815537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 30 01:55:21.815537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: Aug 30 01:55:21.816940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.857818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 816/1629] 10.222.0.50 () {58 vars in 1059 bytes} [Tue Aug 30 01:55:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:21.858901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dc992-55ab-4db3-a20a-f543ac3ffdf3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.872910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dc992-55ab-4db3-a20a-f543ac3ffdf3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:21.879328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:21.880263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:21.880658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:21.881011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] PATH_INFO: `/v3/projects/20b0566541234320ac94388f3c178143/users/86840d0a55c7425b8aa676124cb0ea36/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:21.885900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:21.912200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b0566541234320ac94388f3c178143, user_id=86840d0a55c7425b8aa676124cb0ea36, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:21.915141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:21.937094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 817/1630] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:21 2022] PUT /identity/v3/projects/20b0566541234320ac94388f3c178143/users/86840d0a55c7425b8aa676124cb0ea36/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:21.938116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:21.960542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5da237ee034b3a85285a671d30223c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:22.001387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5da237ee034b3a85285a671d30223c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:22.013874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5da237ee034b3a85285a671d30223c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:22.013874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5da237ee034b3a85285a671d30223c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:22.014825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c11cfb0a-91f5-4095-b346-3ba888a08de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5da237ee034b3a85285a671d30223c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:22.016067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5095f98-c86c-463c-98d9-a681bd17b074 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.025850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.026479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.026758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.027033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.027685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:22.029341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:22.039437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 814/1631] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:55:21 2022] DELETE /identity/v3/users/5b5da237ee034b3a85285a671d30223c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:22.041832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf9f7-1e48-48ff-85d0-f282d2ea3cdb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.045524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf9f7-1e48-48ff-85d0-f282d2ea3cdb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.045675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 818/1632] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:22.045764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf9f7-1e48-48ff-85d0-f282d2ea3cdb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.045878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:22.045878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:22.050651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 815/1633] 10.222.0.50 () {54 vars in 912 bytes} [Tue Aug 30 01:55:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:55:22.052098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84192943-0065-4ba4-a213-237dba0afa45 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.059994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5023b73-851c-4760-884e-c258b26efcaf admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.060577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84192943-0065-4ba4-a213-237dba0afa45 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.060935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84192943-0065-4ba4-a213-237dba0afa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.095340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-84192943-0065-4ba4-a213-237dba0afa45 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:22.098105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-84192943-0065-4ba4-a213-237dba0afa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:22.135858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.136520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.136800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.137377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.137804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:22.139794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:22.152300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 819/1634] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:22.154326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:22.156620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:22.172470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a9ff6-f953-49bc-becf-62313ca49905 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.178057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.178725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.179004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.179275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] PATH_INFO: `/v3/projects/3c1f37bb07df4f17b619198e4327c7b0/users/3dc340e9640b45c5bc586bc304761a18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.203798 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1f37bb07df4f17b619198e4327c7b0, user_id=3dc340e9640b45c5bc586bc304761a18, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:22.213749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:22.229661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 816/1635] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:22.232778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee211163-ef65-462e-9ca2-fc63996c7c90 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.232778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee211163-ef65-462e-9ca2-fc63996c7c90 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.232778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee211163-ef65-462e-9ca2-fc63996c7c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.238958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 820/1636] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:22 2022] PUT /identity/v3/projects/3c1f37bb07df4f17b619198e4327c7b0/users/3dc340e9640b45c5bc586bc304761a18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:22.241999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:22.251975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:22.265978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ee211163-ef65-462e-9ca2-fc63996c7c90 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:22.269674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ee211163-ef65-462e-9ca2-fc63996c7c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:22.360564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:22.378554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c374a8-3df7-4fa4-859e-b69091e4fcf4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.381371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 817/1637] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:55:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:22.383383 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee211163-ef65-462e-9ca2-fc63996c7c90 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:22.393693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.394576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.394694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.394962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] PATH_INFO: `/v3/projects/20b0566541234320ac94388f3c178143/users/86840d0a55c7425b8aa676124cb0ea36/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.397296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b0566541234320ac94388f3c178143, user_id=86840d0a55c7425b8aa676124cb0ea36, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:22.399960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:22.411909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee211163-ef65-462e-9ca2-fc63996c7c90 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.422855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.424856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.425205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.425892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.426765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 821/1638] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:22 2022] PUT /identity/v3/projects/20b0566541234320ac94388f3c178143/users/86840d0a55c7425b8aa676124cb0ea36/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:22.427155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:22.427678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:22.429960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:22.434612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:22.445812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 818/1639] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:22.447202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:22.453990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:22.579216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd002c-b1fd-4c0f-8b03-3478b983793a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.579654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63d2e3-e25e-493f-bd5f-532b10827522 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.591240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f608b-1941-49d1-908a-ef88abcf1d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.591867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f608b-1941-49d1-908a-ef88abcf1d5f service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.594576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f608b-1941-49d1-908a-ef88abcf1d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.594576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f608b-1941-49d1-908a-ef88abcf1d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.729255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.730444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.733051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.733708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.781739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f608b-1941-49d1-908a-ef88abcf1d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:22.786687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f608b-1941-49d1-908a-ef88abcf1d5f service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:22.817614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 822/1640] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:22.822761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2824e3-c73f-461e-9788-e0697f061542 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.823801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2824e3-c73f-461e-9788-e0697f061542 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.825029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2824e3-c73f-461e-9788-e0697f061542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.889609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:22.904089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8d2824e3-c73f-461e-9788-e0697f061542 None None] MFA Rules not processed for user `86840d0a55c7425b8aa676124cb0ea36`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:22.908641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d2824e3-c73f-461e-9788-e0697f061542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:22.915715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:22.917861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:22.944681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 819/1641] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:22.945802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:22.974538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35cfd-dee1-469c-ab5e-9da278a8468e service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:22.987335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4553049-9033-4257-a3e0-75b859488581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:22.988785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553049-9033-4257-a3e0-75b859488581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:22.988917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553049-9033-4257-a3e0-75b859488581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:22.989441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553049-9033-4257-a3e0-75b859488581 admin admin] PATH_INFO: `/v3/projects/3c1f37bb07df4f17b619198e4327c7b0/users/3dc340e9640b45c5bc586bc304761a18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:22.998314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4553049-9033-4257-a3e0-75b859488581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1f37bb07df4f17b619198e4327c7b0, user_id=3dc340e9640b45c5bc586bc304761a18, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:23.005827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4553049-9033-4257-a3e0-75b859488581 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:23.051605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 820/1642] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:22 2022] PUT /identity/v3/projects/3c1f37bb07df4f17b619198e4327c7b0/users/3dc340e9640b45c5bc586bc304761a18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:23.067832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:23.068029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:23.068214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:23.097690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 823/1643] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:23.109024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2824e3-c73f-461e-9788-e0697f061542 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:23.133894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2824e3-c73f-461e-9788-e0697f061542 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:23.160211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] MFA Rules not processed for user `3dc340e9640b45c5bc586bc304761a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:23.162557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:23.178179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:23.254374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:23.255004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:23.255284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:23.255558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:23.259428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:23.301947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 821/1644] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:23.356047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:23.357859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:23.369030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 824/1645] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:55:23.699333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:23.699333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:23.713863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276241f-c636-4952-8ca1-d66576854bbc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:23.735989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:23.738583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:23.738583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:23.738583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:23.780449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:23.789212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:23.819822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 822/1646] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:24.125170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:24.137800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8285aed-c1bd-484e-87ce-255b4975a6a8 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:24.166691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4937fa8-5512-4354-b1e2-f75c79d3d114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:24.168310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4937fa8-5512-4354-b1e2-f75c79d3d114 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:24.168704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4937fa8-5512-4354-b1e2-f75c79d3d114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:24.169084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4937fa8-5512-4354-b1e2-f75c79d3d114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:24.174789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c4937fa8-5512-4354-b1e2-f75c79d3d114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:24.334272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4937fa8-5512-4354-b1e2-f75c79d3d114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:24.336306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4937fa8-5512-4354-b1e2-f75c79d3d114 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:24.348090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 825/1647] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:24.822352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:24.823895 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:24.946779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ecb27-ceff-4158-ba36-5a98cc3ddc13 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:24.957125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:24.957806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:24.958151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:24.958593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] PATH_INFO: `/v3/projects/830abb133bfb4de0a791bf110baab71e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:24.959837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830abb133bfb4de0a791bf110baab71e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:24.962183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:25.047033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:25.058565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830abb133bfb4de0a791bf110baab71e', 'request_id': 'req-6343d9cb-972e-401c-8321-a99de4636681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:25.096452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830abb133bfb4de0a791bf110baab71e', 'request_id': 'req-6343d9cb-972e-401c-8321-a99de4636681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:25.115625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830abb133bfb4de0a791bf110baab71e', 'request_id': 'req-6343d9cb-972e-401c-8321-a99de4636681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:25.130258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] The token cache is being invalidate because project 830abb133bfb4de0a791bf110baab71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:55:25.132556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:25.133345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6343d9cb-972e-401c-8321-a99de4636681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:25.138324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 823/1648] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:55:24 2022] DELETE /identity/v3/projects/830abb133bfb4de0a791bf110baab71e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:25.139978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172e78-b856-4eaa-b44b-1a437eda2be9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.140441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172e78-b856-4eaa-b44b-1a437eda2be9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.140549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172e78-b856-4eaa-b44b-1a437eda2be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.169603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-10172e78-b856-4eaa-b44b-1a437eda2be9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:25.171195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-10172e78-b856-4eaa-b44b-1a437eda2be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:25.257952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 826/1649] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:25.265818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e683-6f8a-4dbe-add0-7e3458a321ec None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.266067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e683-6f8a-4dbe-add0-7e3458a321ec None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.266348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e683-6f8a-4dbe-add0-7e3458a321ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.293142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9694e683-6f8a-4dbe-add0-7e3458a321ec None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:25.294866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9694e683-6f8a-4dbe-add0-7e3458a321ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:25.307888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 824/1650] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:25.313133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10172e78-b856-4eaa-b44b-1a437eda2be9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:25.328970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10172e78-b856-4eaa-b44b-1a437eda2be9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:25.335598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:25.336358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.336753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.337125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.338018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:25.338437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:25.340224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:25.365522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 827/1651] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:25.372003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e683-6f8a-4dbe-add0-7e3458a321ec None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:25.402035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e683-6f8a-4dbe-add0-7e3458a321ec None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:25.414280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:25.416315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.416897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.417376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.418370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:25.421079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:25.469864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 825/1652] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:25 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:55:25.474628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:25.480039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:25.490866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb74e3-dddf-4922-a81a-7e13339627a8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:25.617106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:25.618818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.619286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.620322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.621201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:25.624981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:25.670043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 828/1653] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:25 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:25.676025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:25.692908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44323e60-9436-415e-b947-b8ae48d188f5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:25.698791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:25.699565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.699842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.700330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.701042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:25.702677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:25.733117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 826/1654] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:25.737842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:25.752976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b142b7-9669-4b3d-8801-1d623e511e57 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:25.765273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7b8ba-6f70-43bc-81fb-39652044c1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:25.766280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7b8ba-6f70-43bc-81fb-39652044c1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.766732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7b8ba-6f70-43bc-81fb-39652044c1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.767279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7b8ba-6f70-43bc-81fb-39652044c1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.768380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7b8ba-6f70-43bc-81fb-39652044c1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:25.770630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7b8ba-6f70-43bc-81fb-39652044c1ef admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:25.800214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:25.810723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 829/1655] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:25.811840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425956c6-ecb5-43f8-acb7-49c6459d9b67 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:25.817837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:25.819307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.819593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.819870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] PATH_INFO: `/v3/projects/27de5ef7e8844ea88d6cf717b0da1ebc/users/47e66354f7614713b98d8e6a84a35823/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.841391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27de5ef7e8844ea88d6cf717b0da1ebc, user_id=47e66354f7614713b98d8e6a84a35823, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:25.887964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:25.931746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01eac-4d4f-47a2-b8f1-90aa8a5f6cd1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:25.932114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01eac-4d4f-47a2-b8f1-90aa8a5f6cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:25.932509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01eac-4d4f-47a2-b8f1-90aa8a5f6cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:25.935288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 827/1656] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:25 2022] PUT /identity/v3/projects/27de5ef7e8844ea88d6cf717b0da1ebc/users/47e66354f7614713b98d8e6a84a35823/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:25.979831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-90b01eac-4d4f-47a2-b8f1-90aa8a5f6cd1 None None] MFA Rules not processed for user `47e66354f7614713b98d8e6a84a35823`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:25.981536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-90b01eac-4d4f-47a2-b8f1-90aa8a5f6cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:26.119563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 830/1657] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:26.131635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:26.134898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:26.271717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94561219-d16c-4250-b991-c65171b7d76a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:26.275882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:26.276876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:26.277161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:26.277436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:26.290649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:26.295634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:26.337888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 828/1658] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:29.577856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89c09f-cdd3-4347-bc26-cdcefa10cd00 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:29.578927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89c09f-cdd3-4347-bc26-cdcefa10cd00 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:29.582602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89c09f-cdd3-4347-bc26-cdcefa10cd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:29.677370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2c89c09f-cdd3-4347-bc26-cdcefa10cd00 None None] MFA Rules not processed for user `a06de37941264b979a3bd5aaf84ec73c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:29.678691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2c89c09f-cdd3-4347-bc26-cdcefa10cd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:29.811341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:29.852135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 831/1659] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:29.881758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:29.883415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:29.988437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:30.023782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33eeaf-ec10-4768-b396-b179e4d7949c service glance] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:30.030303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:30.031004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:30.031916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:30.032295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:30.055611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:30.057488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:30.081772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 829/1660] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:30.153212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f232a8e-fec0-4935-b972-67d75694bb6d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:30.153877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f232a8e-fec0-4935-b972-67d75694bb6d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:30.154360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f232a8e-fec0-4935-b972-67d75694bb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:30.244935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9f232a8e-fec0-4935-b972-67d75694bb6d None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:30.247655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9f232a8e-fec0-4935-b972-67d75694bb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:30.406033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 832/1661] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:55:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:30.759490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:30.761222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:30.923012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa530f-8d9b-4640-8974-97d23918c8ef service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:30.937368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354a575-0644-4371-a617-c0d83dc6db9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:30.938244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a575-0644-4371-a617-c0d83dc6db9e service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:30.938680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a575-0644-4371-a617-c0d83dc6db9e service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:30.939361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a575-0644-4371-a617-c0d83dc6db9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:30.950976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354a575-0644-4371-a617-c0d83dc6db9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:30.952814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354a575-0644-4371-a617-c0d83dc6db9e service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:30.992163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 830/1662] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 01:55:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:55:33.390720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ffafd-749d-48f4-810e-9592e93d0491 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:33.391296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ffafd-749d-48f4-810e-9592e93d0491 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:33.391391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ffafd-749d-48f4-810e-9592e93d0491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:33.441054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-529ffafd-749d-48f4-810e-9592e93d0491 None None] MFA Rules not processed for user `a06de37941264b979a3bd5aaf84ec73c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:33.442293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-529ffafd-749d-48f4-810e-9592e93d0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:33.475190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 833/1663] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:35.639151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad0a1f-ff62-4184-87c4-9d7e763c32ee None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:35.639636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad0a1f-ff62-4184-87c4-9d7e763c32ee None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:35.639731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad0a1f-ff62-4184-87c4-9d7e763c32ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:35.755943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f1ad0a1f-ff62-4184-87c4-9d7e763c32ee None None] MFA Rules not processed for user `85d2f3d2a8cb40d38d0ca2dc16609ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:35.757087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f1ad0a1f-ff62-4184-87c4-9d7e763c32ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:35.797109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00411387-4c09-4570-9678-7cb81ecfc537 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:35.798119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00411387-4c09-4570-9678-7cb81ecfc537 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:35.798792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00411387-4c09-4570-9678-7cb81ecfc537 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:35.801386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 834/1664] 10.222.0.50 () {54 vars in 912 bytes} [Tue Aug 30 01:55:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:55:35.820684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:35.822020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:35.825034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:35.980418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] MFA Rules not processed for user `e1c3b7f7c83f48aca27fbd46202f6506`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:35.982242 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:36.050907 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 831/1665] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:36.151339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad0a1f-ff62-4184-87c4-9d7e763c32ee None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:36.179173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 835/1666] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:55:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:55:36.190182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad0a1f-ff62-4184-87c4-9d7e763c32ee None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:36.205894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bc667-9a42-44dd-ba09-b2db617f2b5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:36.208721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bc667-9a42-44dd-ba09-b2db617f2b5d service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:36.209011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bc667-9a42-44dd-ba09-b2db617f2b5d service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:36.209286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bc667-9a42-44dd-ba09-b2db617f2b5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:36.241772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bc667-9a42-44dd-ba09-b2db617f2b5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:36.244917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bc667-9a42-44dd-ba09-b2db617f2b5d service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:36.261548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:36.264261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:36.266518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 832/1667] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:36.287837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915288a-b6ae-4b3d-a735-c682554536ae None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:36.304751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c275c8-1c0c-4859-9f53-37050f778cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:36.309191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c275c8-1c0c-4859-9f53-37050f778cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:36.309666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c275c8-1c0c-4859-9f53-37050f778cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:36.310104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c275c8-1c0c-4859-9f53-37050f778cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:36.347514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c275c8-1c0c-4859-9f53-37050f778cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:36.353937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c275c8-1c0c-4859-9f53-37050f778cc2 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:36.377420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 836/1668] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:36.477806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1baf6-34e7-457f-a065-77a08ff14f00 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:36.478473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1baf6-34e7-457f-a065-77a08ff14f00 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:36.478898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1baf6-34e7-457f-a065-77a08ff14f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:36.538537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c2c1baf6-34e7-457f-a065-77a08ff14f00 None None] MFA Rules not processed for user `cc7381cfffee4842bee1950a18caf807`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:36.539807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c2c1baf6-34e7-457f-a065-77a08ff14f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:36.577406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:36.696041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 833/1669] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:36.710140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:36.710427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:36.710706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:36.747084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] MFA Rules not processed for user `cc7381cfffee4842bee1950a18caf807`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:36.748650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:36.817561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 837/1670] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:36.827922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1baf6-34e7-457f-a065-77a08ff14f00 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:36.843991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1baf6-34e7-457f-a065-77a08ff14f00 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:36.855725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:36.857296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:36.858042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:36.858480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:36.900281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:36.906752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:36.925942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 834/1671] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:55:36.949936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:36.951737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:37.036884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:37.063456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711d023-a109-4f72-ad57-b8f64669766a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:37.077905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474364ae-e28c-44d2-8914-55d90d229c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:37.078541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474364ae-e28c-44d2-8914-55d90d229c87 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:37.078824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474364ae-e28c-44d2-8914-55d90d229c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:37.079269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474364ae-e28c-44d2-8914-55d90d229c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:37.082525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-474364ae-e28c-44d2-8914-55d90d229c87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:37.250940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474364ae-e28c-44d2-8914-55d90d229c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:37.258803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474364ae-e28c-44d2-8914-55d90d229c87 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:37.285886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 838/1672] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:38.171959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:38.173574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:38.323151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0c6eb-2d96-4d1a-acdf-08c3c6f58c2b service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:38.329481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0d4fe-6d16-437e-aafc-292249454ab5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:38.330276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d4fe-6d16-437e-aafc-292249454ab5 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:38.330773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d4fe-6d16-437e-aafc-292249454ab5 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:38.331254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d4fe-6d16-437e-aafc-292249454ab5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:38.335366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6cd0d4fe-6d16-437e-aafc-292249454ab5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:38.352878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0d4fe-6d16-437e-aafc-292249454ab5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:38.355151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0d4fe-6d16-437e-aafc-292249454ab5 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:38.383787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 835/1673] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:38.401632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306f03-da83-4638-b21a-7c390d977d75 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:38.402130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306f03-da83-4638-b21a-7c390d977d75 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:38.402499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306f03-da83-4638-b21a-7c390d977d75 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:38.404055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 839/1674] 10.222.0.50 () {54 vars in 912 bytes} [Tue Aug 30 01:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:55:38.411164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5096c-1422-4249-b06d-0910d52ccc24 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:38.411658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5096c-1422-4249-b06d-0910d52ccc24 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:38.412075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5096c-1422-4249-b06d-0910d52ccc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:38.463708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-faa5096c-1422-4249-b06d-0910d52ccc24 None None] MFA Rules not processed for user `e1c3b7f7c83f48aca27fbd46202f6506`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:38.466811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-faa5096c-1422-4249-b06d-0910d52ccc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:38.512277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4306f03-da83-4638-b21a-7c390d977d75 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:38.514738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e4306f03-da83-4638-b21a-7c390d977d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:38.531829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 836/1675] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:55:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:38.534734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bea78-3916-4e44-aff7-db0481d70f0a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:38.535026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bea78-3916-4e44-aff7-db0481d70f0a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:38.535309 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bea78-3916-4e44-aff7-db0481d70f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:38.540947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4306f03-da83-4638-b21a-7c390d977d75 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:38.548004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:38.549815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:38.550326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:38.550721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:38.560758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:38.562395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:38.605083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 840/1676] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:55:38.627558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7e2bea78-3916-4e44-aff7-db0481d70f0a None None] MFA Rules not processed for user `68fcefa4a0db4bd6a051b1446e1a0d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:38.628880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7e2bea78-3916-4e44-aff7-db0481d70f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:38.730816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 837/1677] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:38.751593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:38.753139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:38.779941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c785dc-ca71-49d5-b1a2-44976ab698be service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:38.826218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42a9cc-30f3-4dad-9fad-2c0438e3a986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:38.827165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42a9cc-30f3-4dad-9fad-2c0438e3a986 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:38.827582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42a9cc-30f3-4dad-9fad-2c0438e3a986 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:38.828028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42a9cc-30f3-4dad-9fad-2c0438e3a986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:38.846363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42a9cc-30f3-4dad-9fad-2c0438e3a986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:38.848398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42a9cc-30f3-4dad-9fad-2c0438e3a986 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:38.876364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 841/1678] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:55:41.269850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cb76f-421c-4d93-884a-22c0ab4f7687 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:41.270876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cb76f-421c-4d93-884a-22c0ab4f7687 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:41.271315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cb76f-421c-4d93-884a-22c0ab4f7687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:41.354058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1b5cb76f-421c-4d93-884a-22c0ab4f7687 None None] MFA Rules not processed for user `85d2f3d2a8cb40d38d0ca2dc16609ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:41.355296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1b5cb76f-421c-4d93-884a-22c0ab4f7687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:41.390467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 838/1679] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:41.972127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:41.972653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:41.972653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:42.009256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] MFA Rules not processed for user `68fcefa4a0db4bd6a051b1446e1a0d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:42.016410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:42.033280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 842/1680] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:46.929692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452e022-534d-49fd-b51d-661a5ef44197 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:46.930170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452e022-534d-49fd-b51d-661a5ef44197 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:46.930242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452e022-534d-49fd-b51d-661a5ef44197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:46.957610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:46.957610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:46.959225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:46.966942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0452e022-534d-49fd-b51d-661a5ef44197 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:46.968544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0452e022-534d-49fd-b51d-661a5ef44197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:46.979232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 839/1681] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:55:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:46.997655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452e022-534d-49fd-b51d-661a5ef44197 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:47.021592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452e022-534d-49fd-b51d-661a5ef44197 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:47.046438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb260f7-a976-46fb-be35-cca8af375af2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:47.047117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb260f7-a976-46fb-be35-cca8af375af2 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:47.047399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb260f7-a976-46fb-be35-cca8af375af2 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:47.047674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb260f7-a976-46fb-be35-cca8af375af2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:47.068140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb260f7-a976-46fb-be35-cca8af375af2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:47.072410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb260f7-a976-46fb-be35-cca8af375af2 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:47.073820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290be0f-7aa2-48dc-8e0f-e107ea55435a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:47.084341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:47.085250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:47.086988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:47.097869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] PATH_INFO: `/v3/users/b74a7caa5628414bbbf9bab161625f0d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:47.099453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74a7caa5628414bbbf9bab161625f0d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:47.103910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 840/1682] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 01:55:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:55:47.105781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:47.236723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74a7caa5628414bbbf9bab161625f0d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:47.275680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74a7caa5628414bbbf9bab161625f0d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:47.285115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:47.297223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74a7caa5628414bbbf9bab161625f0d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:47.298943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74a7caa5628414bbbf9bab161625f0d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:47.329017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74a7caa5628414bbbf9bab161625f0d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:47.347515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 843/1683] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:55:46 2022] DELETE /identity/v3/users/b74a7caa5628414bbbf9bab161625f0d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:47.351556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:47.352034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:47.352513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:47.400670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:47.408853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:47.619037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 841/1684] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:47.631290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:47.633300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:47.806807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f2c7-59b7-4988-86e1-a8a3b0a5a478 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:47.817988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f795-0ddd-42f8-828d-5b5545355341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:47.824359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f795-0ddd-42f8-828d-5b5545355341 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:47.824359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f795-0ddd-42f8-828d-5b5545355341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:47.824359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f795-0ddd-42f8-828d-5b5545355341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:47.867776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f795-0ddd-42f8-828d-5b5545355341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:47.867776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f795-0ddd-42f8-828d-5b5545355341 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:47.944747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 844/1685] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:49.414206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:49.415679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:49.474463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db52f2-6993-4379-abd5-c5b5b4f1fe22 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:49.510642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:49.511297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:49.516380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:49.516920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] PATH_INFO: `/v3/projects/56a61719b5c8402b9542dacad4242644` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:49.520852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56a61719b5c8402b9542dacad4242644)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:49.534101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:49.721755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56a61719b5c8402b9542dacad4242644', 'request_id': 'req-f75b9eb6-9abc-4449-903c-9b689e905722'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:49.754245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56a61719b5c8402b9542dacad4242644', 'request_id': 'req-f75b9eb6-9abc-4449-903c-9b689e905722'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:49.766215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56a61719b5c8402b9542dacad4242644', 'request_id': 'req-f75b9eb6-9abc-4449-903c-9b689e905722'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:49.800952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] The token cache is being invalidate because project 56a61719b5c8402b9542dacad4242644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:55:49.801208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:49.801957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:49.803756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 842/1686] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:55:49 2022] DELETE /identity/v3/projects/56a61719b5c8402b9542dacad4242644 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:49.810372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024f037-2d3a-47af-a891-eac890d9be20 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:49.810884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024f037-2d3a-47af-a891-eac890d9be20 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:49.810884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024f037-2d3a-47af-a891-eac890d9be20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:49.851987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2024f037-2d3a-47af-a891-eac890d9be20 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:49.853521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2024f037-2d3a-47af-a891-eac890d9be20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:49.974909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:49.976689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:49.987656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 845/1687] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:49.989374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22ab73-9034-4c97-9672-6b9346b05b34 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:49.989654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22ab73-9034-4c97-9672-6b9346b05b34 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:49.989936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22ab73-9034-4c97-9672-6b9346b05b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.011971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b9eb6-9abc-4449-903c-9b689e905722 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.026935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.027584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.027866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.028319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] PATH_INFO: `/v3/users/e83d4a4186a649899593fa5443425147` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.029751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e83d4a4186a649899593fa5443425147)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.030000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fe22ab73-9034-4c97-9672-6b9346b05b34 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:50.031092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fe22ab73-9034-4c97-9672-6b9346b05b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:50.031735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.054245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 846/1688] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:50.055966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22ab73-9034-4c97-9672-6b9346b05b34 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.069999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22ab73-9034-4c97-9672-6b9346b05b34 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.077534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.078571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.078959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.079397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.080233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:50.080658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.082969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.105037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 847/1689] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:50.109008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.109484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e83d4a4186a649899593fa5443425147'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:50.121855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f3800-a85e-4869-a768-e27b6d779783 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.128115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.128874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.129223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.129573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.130404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.132569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.140045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e83d4a4186a649899593fa5443425147'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:50.142725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:50.158903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e83d4a4186a649899593fa5443425147'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:50.166670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e83d4a4186a649899593fa5443425147'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:50.167107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 848/1690] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:50 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:55:50.170613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1310f8b0-eb59-42ba-b734-5ef5381b8a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e83d4a4186a649899593fa5443425147'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:50.170860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.173912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:50.201892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 843/1691] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:55:49 2022] DELETE /identity/v3/users/e83d4a4186a649899593fa5443425147 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:50.203715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.204070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.204561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.244275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:50.245848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:50.265426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:50.292748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794e91d-fdbd-451d-955c-2beb40acc7df admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.334397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 844/1692] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:50.344934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.349778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:50.362894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.363531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.363824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.364306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.364896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.366905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.392993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 849/1693] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:50 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:50.396375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.408568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f354a02-e900-4d55-b649-d16532b65e7c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.413506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.414133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.414410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.414684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.415333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.416996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.427529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e0121-53c3-4a47-8e55-c25900ec798a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.428341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 850/1694] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:55:50.432259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.442977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247a1d1-8622-4ed3-a529-48a371d9a918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.443997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247a1d1-8622-4ed3-a529-48a371d9a918 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.444315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247a1d1-8622-4ed3-a529-48a371d9a918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.444793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247a1d1-8622-4ed3-a529-48a371d9a918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.449533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a3ec1-5d4a-4860-a917-444e9a516555 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.455920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.456646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.457023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.457353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.458074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.458605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247a1d1-8622-4ed3-a529-48a371d9a918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.459890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.460456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247a1d1-8622-4ed3-a529-48a371d9a918 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.472281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 851/1695] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:50.472864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 845/1696] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:55:50.476071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.486878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad1e47-68ae-409f-a68c-efaa26e99d2c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.493110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.493805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.494167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.494604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] PATH_INFO: `/v3/projects/80d31576169349449039f7766167b1f3/users/64614891540a4f79913d71672b081781/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.512708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d31576169349449039f7766167b1f3, user_id=64614891540a4f79913d71672b081781, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.515415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.537707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 852/1697] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:50 2022] PUT /identity/v3/projects/80d31576169349449039f7766167b1f3/users/64614891540a4f79913d71672b081781/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:50.551686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.551997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.552718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.630226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] MFA Rules not processed for user `64614891540a4f79913d71672b081781`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:50.631787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:50.730303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 846/1698] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:55:50.735408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.842150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5650f7-3774-4148-b431-af02cc837e61 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:50.848064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:50.850431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:50.850431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:50.850431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:50.850431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:50.852976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:50.879582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.886328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:50.952770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 853/1699] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:50 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:50.954268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:50.956849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666928c-3719-4838-8ec3-0ab56747bb6f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.003122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:51.016177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:51.081642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc11c61-5f0f-4ec3-8d4b-528abb1b42c9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.087476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.088020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.088325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.088641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.089292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.091392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.100555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.101820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.102208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.102989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] PATH_INFO: `/v3/projects/4ad56c2e3e7b4d35831b3c8f72db6fcc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.104839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad56c2e3e7b4d35831b3c8f72db6fcc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.107213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.117784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 854/1700] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:50 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:51.122101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:51.133400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947ef68-bf9e-4779-ab05-79703a459d45 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.149896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.150656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.150859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.151124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.151773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.153621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.164839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 855/1701] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:51.169032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:51.175520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad56c2e3e7b4d35831b3c8f72db6fcc', 'request_id': 'req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.180143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fad1eb-748b-4757-9628-a66abc2fc2e4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.187743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad56c2e3e7b4d35831b3c8f72db6fcc', 'request_id': 'req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.195171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad56c2e3e7b4d35831b3c8f72db6fcc', 'request_id': 'req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.206288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] The token cache is being invalidate because project 4ad56c2e3e7b4d35831b3c8f72db6fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:55:51.206576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.207237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.209083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 847/1702] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:55:50 2022] DELETE /identity/v3/projects/4ad56c2e3e7b4d35831b3c8f72db6fcc => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:55:51.212734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:51.214350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:51.283477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.284708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.285485 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.285769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] PATH_INFO: `/v3/projects/00f1862eb58f493c9f8a29bf77d52a0e/users/d5ea918c23ad47bf98f147b5e0a386ff/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.318777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e5547-cbdd-4b0f-bd6e-bd85dd761efe admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.325515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.326393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.326748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.327099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] PATH_INFO: `/v3/users/3dc340e9640b45c5bc586bc304761a18` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.328571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc340e9640b45c5bc586bc304761a18)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.330987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f1862eb58f493c9f8a29bf77d52a0e, user_id=d5ea918c23ad47bf98f147b5e0a386ff, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.331110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.333972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.358660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 856/1703] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:51 2022] PUT /identity/v3/projects/00f1862eb58f493c9f8a29bf77d52a0e/users/d5ea918c23ad47bf98f147b5e0a386ff/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:55:51.375454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:51.379609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:51.410688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc340e9640b45c5bc586bc304761a18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.411875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:51.430937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc340e9640b45c5bc586bc304761a18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.439772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc340e9640b45c5bc586bc304761a18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.440558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc340e9640b45c5bc586bc304761a18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.441245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc340e9640b45c5bc586bc304761a18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:51.458340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 848/1704] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:55:51 2022] DELETE /identity/v3/users/3dc340e9640b45c5bc586bc304761a18 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:51.465662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c13243-ceb6-4217-82f6-28547ab357be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.488981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:51.549576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebea427-c9c5-464f-94a4-b7ccef44a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.550192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea427-c9c5-464f-94a4-b7ccef44a9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.550466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea427-c9c5-464f-94a4-b7ccef44a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.550751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea427-c9c5-464f-94a4-b7ccef44a9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.551383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebea427-c9c5-464f-94a4-b7ccef44a9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.553040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebea427-c9c5-464f-94a4-b7ccef44a9c1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.563531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 857/1705] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:55:51.568933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:51.570456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:51.593221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d02746-8d8d-4db7-b396-74485eba3001 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.599108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.599859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.600142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.600468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] PATH_INFO: `/v3/projects/00f1862eb58f493c9f8a29bf77d52a0e/users/d5ea918c23ad47bf98f147b5e0a386ff/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.602473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f1862eb58f493c9f8a29bf77d52a0e, user_id=d5ea918c23ad47bf98f147b5e0a386ff, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.605210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.629179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 849/1706] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:51 2022] PUT /identity/v3/projects/00f1862eb58f493c9f8a29bf77d52a0e/users/d5ea918c23ad47bf98f147b5e0a386ff/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:51.639195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.639512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.639789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.679663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] MFA Rules not processed for user `d5ea918c23ad47bf98f147b5e0a386ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:51.680928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:51.766966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 858/1707] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:51.781620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:51.788325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:51.809216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:51.810856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:51.916962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3277a-7eab-49a0-ac4a-48e2d1ebe469 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.930774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa4386-0bd2-4360-a249-c9a4f1200baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.931412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa4386-0bd2-4360-a249-c9a4f1200baa service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.931694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa4386-0bd2-4360-a249-c9a4f1200baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.931969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa4386-0bd2-4360-a249-c9a4f1200baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.941177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af0f5-ae8b-489c-acdb-5b5da0ccdf6a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:51.943134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa4386-0bd2-4360-a249-c9a4f1200baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.944779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa4386-0bd2-4360-a249-c9a4f1200baa service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.947435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:51.948409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:51.948998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:51.949342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] PATH_INFO: `/v3/projects/3c1f37bb07df4f17b619198e4327c7b0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:51.950756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1f37bb07df4f17b619198e4327c7b0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:51.951921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:55:51.951921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Aug 30 01:55:51.951921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: Aug 30 01:55:51.953340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:51.955745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 850/1708] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:52.043276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:52.045001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1f37bb07df4f17b619198e4327c7b0', 'request_id': 'req-6b65e7a9-4acb-4824-8413-b7da086b6cd8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:52.081342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1f37bb07df4f17b619198e4327c7b0', 'request_id': 'req-6b65e7a9-4acb-4824-8413-b7da086b6cd8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:52.089189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1f37bb07df4f17b619198e4327c7b0', 'request_id': 'req-6b65e7a9-4acb-4824-8413-b7da086b6cd8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:52.132806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] The token cache is being invalidate because project 3c1f37bb07df4f17b619198e4327c7b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:55:52.133391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:52.134344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:55:52.138838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 859/1709] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:55:51 2022] DELETE /identity/v3/projects/3c1f37bb07df4f17b619198e4327c7b0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:52.141710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be76a7-ec90-431d-9a6e-f1e6c5eb6bc7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.142039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be76a7-ec90-431d-9a6e-f1e6c5eb6bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.142363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be76a7-ec90-431d-9a6e-f1e6c5eb6bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.147319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:52.163743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:52.213711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-49be76a7-ec90-431d-9a6e-f1e6c5eb6bc7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:52.216679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-49be76a7-ec90-431d-9a6e-f1e6c5eb6bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:52.302804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e7a9-4acb-4824-8413-b7da086b6cd8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:52.310926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:52.311704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.312433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.312433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.315367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:52.316075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 851/1710] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:52.325391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceb62e-aa9c-43e1-ac73-aaca2db02d98 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.325687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceb62e-aa9c-43e1-ac73-aaca2db02d98 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.325971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceb62e-aa9c-43e1-ac73-aaca2db02d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.361670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fdceb62e-aa9c-43e1-ac73-aaca2db02d98 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:52.363741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fdceb62e-aa9c-43e1-ac73-aaca2db02d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:52.379971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 852/1711] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:55:52.381660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceb62e-aa9c-43e1-ac73-aaca2db02d98 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:52.409348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceb62e-aa9c-43e1-ac73-aaca2db02d98 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:52.431436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:52.433168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.433946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.434911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.435757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:55:52.437301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:52.440691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:52.457583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:52.462537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:52.469688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 853/1712] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:55:52.471764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:52.479743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 860/1713] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:52.496281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e11b3-0458-4e6f-a5cf-bacac0918362 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:52.510333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:52.510973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.511242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.511517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.512421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:52.514618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:52.580543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 854/1714] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:52 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:55:52.585198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:52.599400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f83fa1-eddb-4bc9-9cee-251287f36edf service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:52.680551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:52.681259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.681737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.682082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.682797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:52.684960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:52.726984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 861/1715] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:52 2022] POST /identity/v3/users => generated 279 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:52.729855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:52.748232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61387c6d-2a58-43b6-8a2a-f78ba3930d4d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:52.758874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:52.759501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.759825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.760116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.760823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:52.762419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:52.773853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 855/1716] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:52.777908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:52.788675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de0c25-f481-4a92-a9cd-b9822a5f2f5f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:52.793294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09997dd8-de15-4ddc-a5ea-b4057e0485d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:52.793899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09997dd8-de15-4ddc-a5ea-b4057e0485d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.794179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09997dd8-de15-4ddc-a5ea-b4057e0485d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.794613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09997dd8-de15-4ddc-a5ea-b4057e0485d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.795098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09997dd8-de15-4ddc-a5ea-b4057e0485d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:52.796742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09997dd8-de15-4ddc-a5ea-b4057e0485d0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:52.807021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 862/1717] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:52.811284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:52.816553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:52.823123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d35c2c-4f96-411e-89d2-fbd045a86c84 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:52.828961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:52.830125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.830406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.830711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] PATH_INFO: `/v3/projects/a3a4ca5a7c7a42eb98e02132564f0d11/users/162c210af61b4793b110700bb6cf84a3/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.845234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a4ca5a7c7a42eb98e02132564f0d11, user_id=162c210af61b4793b110700bb6cf84a3, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:52.848562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:52.861196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 856/1718] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:52 2022] PUT /identity/v3/projects/a3a4ca5a7c7a42eb98e02132564f0d11/users/162c210af61b4793b110700bb6cf84a3/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:55:52.871333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101acca0-9bbb-486f-8fef-7ace3f06de0b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:52.871564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101acca0-9bbb-486f-8fef-7ace3f06de0b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:52.871839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101acca0-9bbb-486f-8fef-7ace3f06de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:52.892671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:52.902954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-101acca0-9bbb-486f-8fef-7ace3f06de0b None None] MFA Rules not processed for user `162c210af61b4793b110700bb6cf84a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:52.904226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-101acca0-9bbb-486f-8fef-7ace3f06de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:52.969712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 863/1719] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:52.974245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.039639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766f727-2b6a-490b-b0eb-fe98bc12f71a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.044462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.045065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.045348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.045626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.046270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.048155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.074586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 857/1720] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:55:52 2022] POST /identity/v3/projects => generated 353 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:53.078785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101acca0-9bbb-486f-8fef-7ace3f06de0b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.090031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101acca0-9bbb-486f-8fef-7ace3f06de0b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.144682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.145220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.145497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.145771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.146423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.148440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.168888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 864/1721] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:55:53 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:55:53.172863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.183637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fde17-3b8d-473a-b39e-e125128bdaa7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.188118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.188736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.189754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.190027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.190655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.192587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.202944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 858/1722] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:53.208164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.220450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33554c1-f47d-4de6-85d7-c881892e0c83 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.222908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.223560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.223831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.224100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] PATH_INFO: `/v3/projects/d9464bfe3a214a26b6abf04da53759cc/users/26f5347868f94f2f901bf1ef0526286e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.238369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9464bfe3a214a26b6abf04da53759cc, user_id=26f5347868f94f2f901bf1ef0526286e, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.241066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.255412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 865/1723] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:53 2022] PUT /identity/v3/projects/d9464bfe3a214a26b6abf04da53759cc/users/26f5347868f94f2f901bf1ef0526286e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:53.260177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.270744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211011d-0913-4b5a-a9f2-92348a925e88 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.331349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08932c91-5ec3-4d81-88f9-fbe28ba3c723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.331973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932c91-5ec3-4d81-88f9-fbe28ba3c723 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.332267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932c91-5ec3-4d81-88f9-fbe28ba3c723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.332562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932c91-5ec3-4d81-88f9-fbe28ba3c723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.333204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08932c91-5ec3-4d81-88f9-fbe28ba3c723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.334806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08932c91-5ec3-4d81-88f9-fbe28ba3c723 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.345114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 859/1724] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:55:53.349337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.364565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331004fc-c4d8-4245-9c94-49db60cbf276 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.369132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.369778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.370055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.370328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] PATH_INFO: `/v3/projects/d9464bfe3a214a26b6abf04da53759cc/users/26f5347868f94f2f901bf1ef0526286e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.372188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9464bfe3a214a26b6abf04da53759cc, user_id=26f5347868f94f2f901bf1ef0526286e, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.374843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.385538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 866/1725] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:55:53 2022] PUT /identity/v3/projects/d9464bfe3a214a26b6abf04da53759cc/users/26f5347868f94f2f901bf1ef0526286e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:55:53.396580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f2dd1-3d1d-419c-b1bd-2831528aff23 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.396812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f2dd1-3d1d-419c-b1bd-2831528aff23 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.397090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f2dd1-3d1d-419c-b1bd-2831528aff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.424390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-843f2dd1-3d1d-419c-b1bd-2831528aff23 None None] MFA Rules not processed for user `26f5347868f94f2f901bf1ef0526286e`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:55:53.425580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-843f2dd1-3d1d-419c-b1bd-2831528aff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:55:53.462409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:53.492321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 860/1726] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:55:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:55:53.504252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.562417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c27a0-aad5-4d6b-b838-2ef44309ef6f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.566784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db5b89-3233-42c1-a575-6a59ad42037f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.567399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db5b89-3233-42c1-a575-6a59ad42037f service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.567677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db5b89-3233-42c1-a575-6a59ad42037f service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.567950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db5b89-3233-42c1-a575-6a59ad42037f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.574513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:55:53.578255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db5b89-3233-42c1-a575-6a59ad42037f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.580064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db5b89-3233-42c1-a575-6a59ad42037f service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.594835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 867/1727] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:55:53.683609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f2dd1-3d1d-419c-b1bd-2831528aff23 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:55:53.693285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f2dd1-3d1d-419c-b1bd-2831528aff23 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:55:53.697315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:55:53.697852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:55:53.698135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:55:53.698406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:55:53.770740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:55:53.772202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:55:53.781666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 861/1728] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:55:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:00.325020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d973f-0f7b-41c9-8f44-58e2d51b9fab None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:00.325474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d973f-0f7b-41c9-8f44-58e2d51b9fab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:00.325749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d973f-0f7b-41c9-8f44-58e2d51b9fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:00.357777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ca5d973f-0f7b-41c9-8f44-58e2d51b9fab None None] MFA Rules not processed for user `e1c3b7f7c83f48aca27fbd46202f6506`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:00.366624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ca5d973f-0f7b-41c9-8f44-58e2d51b9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:00.474381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 868/1729] 10.222.0.57 () {58 vars in 1015 bytes} [Tue Aug 30 01:56:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:00.646995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:00.648224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:00.676008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6f462-cf81-49ee-830a-15a843280e13 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:00.685627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a87868-7134-4be2-9892-0e086a666cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:00.686516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a87868-7134-4be2-9892-0e086a666cca service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:00.686798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a87868-7134-4be2-9892-0e086a666cca service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:00.687091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a87868-7134-4be2-9892-0e086a666cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:00.711308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a87868-7134-4be2-9892-0e086a666cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:00.713318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a87868-7134-4be2-9892-0e086a666cca service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:00.723869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 862/1730] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:02.556019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b3fb-8c54-40f9-963f-58bf5d5845ba None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:02.556477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b3fb-8c54-40f9-963f-58bf5d5845ba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:02.556797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b3fb-8c54-40f9-963f-58bf5d5845ba None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:02.558539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 869/1731] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 01:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:56:02.569671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6389f9-7443-499d-bcbd-99a6eaeed3b1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:02.571202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6389f9-7443-499d-bcbd-99a6eaeed3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:02.574195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6389f9-7443-499d-bcbd-99a6eaeed3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:02.622306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5b6389f9-7443-499d-bcbd-99a6eaeed3b1 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:02.623523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5b6389f9-7443-499d-bcbd-99a6eaeed3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:02.759456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 863/1732] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 01:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:02.823048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b3fb-8c54-40f9-963f-58bf5d5845ba None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:02.824898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2a74b3fb-8c54-40f9-963f-58bf5d5845ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:02.854065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b3fb-8c54-40f9-963f-58bf5d5845ba None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:02.869305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595358d-ce92-432e-a813-e71d44d29864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:02.870371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595358d-ce92-432e-a813-e71d44d29864 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:02.870371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595358d-ce92-432e-a813-e71d44d29864 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:02.870568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595358d-ce92-432e-a813-e71d44d29864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:02.881233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595358d-ce92-432e-a813-e71d44d29864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:02.883004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595358d-ce92-432e-a813-e71d44d29864 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:02.896647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 870/1733] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:56:03.450934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859473-7692-49ff-9df3-ac07039ac375 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:03.451220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859473-7692-49ff-9df3-ac07039ac375 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:03.451500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb859473-7692-49ff-9df3-ac07039ac375 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:03.452905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 864/1734] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 01:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:56:03.458904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc536-c36b-4375-b833-ea0d40101b3e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:03.459191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc536-c36b-4375-b833-ea0d40101b3e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:03.459470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc536-c36b-4375-b833-ea0d40101b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:03.478789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0c5dc536-c36b-4375-b833-ea0d40101b3e None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:03.479992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0c5dc536-c36b-4375-b833-ea0d40101b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:03.489212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 871/1735] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 01:56:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:03.501021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859473-7692-49ff-9df3-ac07039ac375 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:03.515567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb859473-7692-49ff-9df3-ac07039ac375 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:03.521566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b5217-ecd4-43b8-99d7-167d32683ccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:03.522161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b5217-ecd4-43b8-99d7-167d32683ccb service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:03.522436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b5217-ecd4-43b8-99d7-167d32683ccb service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:03.522707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b5217-ecd4-43b8-99d7-167d32683ccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:03.531920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b5217-ecd4-43b8-99d7-167d32683ccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:03.533049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b5217-ecd4-43b8-99d7-167d32683ccb service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:03.543179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 865/1736] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:13.473683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=35 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:13.539890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:21.635854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f6e8-2353-41f9-9b58-420380ec555a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:21.636495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f6e8-2353-41f9-9b58-420380ec555a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:21.636744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f6e8-2353-41f9-9b58-420380ec555a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:21.638924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 872/1737] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 01:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:56:21.649366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:21.649855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:21.649952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:21.675532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:21.677246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:21.686167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 866/1738] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:21.700038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f6e8-2353-41f9-9b58-420380ec555a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:21.714419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f6e8-2353-41f9-9b58-420380ec555a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:21.720016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f8ded-64bc-4d99-8725-0a69c5af9461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:21.721165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8ded-64bc-4d99-8725-0a69c5af9461 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:21.721444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8ded-64bc-4d99-8725-0a69c5af9461 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:21.721720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8ded-64bc-4d99-8725-0a69c5af9461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:21.730517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f8ded-64bc-4d99-8725-0a69c5af9461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:21.732089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f8ded-64bc-4d99-8725-0a69c5af9461 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:21.742876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 873/1739] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:24.485026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:24.486407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:24.549995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073d620-d8fd-47ac-a86f-2da9fa522dde None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:24.555482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:24.556417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:24.556842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:24.557125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] PATH_INFO: `/v3/users/b6bd8657bf5f4dc18cc0ead8e501a176` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:24.558286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6bd8657bf5f4dc18cc0ead8e501a176)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:24.560454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:24.625728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6bd8657bf5f4dc18cc0ead8e501a176'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:24.644769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6bd8657bf5f4dc18cc0ead8e501a176'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:24.657262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6bd8657bf5f4dc18cc0ead8e501a176'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:24.658139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6bd8657bf5f4dc18cc0ead8e501a176'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:24.658895 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6bd8657bf5f4dc18cc0ead8e501a176'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:24.672597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 867/1740] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:24 2022] DELETE /identity/v3/users/b6bd8657bf5f4dc18cc0ead8e501a176 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:24.677760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:24.678611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:24.679296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:24.715407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:24.716805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:24.814816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 874/1741] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:24.824684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:24.826306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:24.896355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff5beb-f1ce-4d85-a421-ad156ce05a4d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:24.899997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:24.900645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:24.900924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:24.901820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:24.912342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:24.914256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:24.925017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 868/1742] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:25.348341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:25.349920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:25.364621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d6125-9752-4a0c-a56b-16cde3aaee5b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:25.369636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:25.370279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:25.370573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:25.371044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] PATH_INFO: `/v3/projects/abac996e90424840a7e39b2620b838d7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:25.372381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abac996e90424840a7e39b2620b838d7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:25.375044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:25.441502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abac996e90424840a7e39b2620b838d7', 'request_id': 'req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.454054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abac996e90424840a7e39b2620b838d7', 'request_id': 'req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.465342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abac996e90424840a7e39b2620b838d7', 'request_id': 'req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.484870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] The token cache is being invalidate because project abac996e90424840a7e39b2620b838d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:25.485224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.485872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.487462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 875/1743] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:25 2022] DELETE /identity/v3/projects/abac996e90424840a7e39b2620b838d7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:25.492494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:25.509243 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279f2d6-3f74-45d3-be26-6d296c209978 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:25.533029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:25.602103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:25.602745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:25.603022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:25.603298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] PATH_INFO: `/v3/users/86840d0a55c7425b8aa676124cb0ea36` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:25.604624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86840d0a55c7425b8aa676124cb0ea36)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:25.606732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:25.652122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86840d0a55c7425b8aa676124cb0ea36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.664711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86840d0a55c7425b8aa676124cb0ea36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.671369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86840d0a55c7425b8aa676124cb0ea36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.672032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86840d0a55c7425b8aa676124cb0ea36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.672665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b2ee67a1-3ba9-440c-92ee-b4d7f2d1c739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86840d0a55c7425b8aa676124cb0ea36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:25.685983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 869/1744] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:25 2022] DELETE /identity/v3/users/86840d0a55c7425b8aa676124cb0ea36 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:26.086850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.089153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:26.109033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c28f8a-8386-4a3c-8f8d-2e8369aeda45 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:26.195299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:26.196041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.196358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.196635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] PATH_INFO: `/v3/projects/20b0566541234320ac94388f3c178143` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.197909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b0566541234320ac94388f3c178143)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:26.200638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:26.212935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:26.253646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b0566541234320ac94388f3c178143', 'request_id': 'req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:26.261427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b0566541234320ac94388f3c178143', 'request_id': 'req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:26.268040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b0566541234320ac94388f3c178143', 'request_id': 'req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:26.281675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] The token cache is being invalidate because project 20b0566541234320ac94388f3c178143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:26.281967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:26.282627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e0cbb689-48db-4aa5-a8cf-35e49daa4be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:26.287607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 876/1745] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:26 2022] DELETE /identity/v3/projects/20b0566541234320ac94388f3c178143 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:26.289093 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763da720-7dfc-4760-8b91-f82fb96e797a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.289381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763da720-7dfc-4760-8b91-f82fb96e797a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.289843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763da720-7dfc-4760-8b91-f82fb96e797a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.329767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-763da720-7dfc-4760-8b91-f82fb96e797a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:26.331591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-763da720-7dfc-4760-8b91-f82fb96e797a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:26.429532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 870/1746] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:26.437839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fed70-e529-4096-9734-cb9cda8777fc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.438119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fed70-e529-4096-9734-cb9cda8777fc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.438466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fed70-e529-4096-9734-cb9cda8777fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.461206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d11fed70-e529-4096-9734-cb9cda8777fc None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:26.463482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d11fed70-e529-4096-9734-cb9cda8777fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:26.474737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 877/1747] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:26.478783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763da720-7dfc-4760-8b91-f82fb96e797a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.494305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763da720-7dfc-4760-8b91-f82fb96e797a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:26.505897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:26.506504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.506778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.507259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.508096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:56:26.508769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:26.511105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:26.528318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fed70-e529-4096-9734-cb9cda8777fc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.534338 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 871/1748] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:56:26.538682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fed70-e529-4096-9734-cb9cda8777fc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:26.544180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:26.544894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.545173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.545440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.546085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:26.547983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:26.578528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 878/1749] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:26 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:26.583154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.593417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20202561-1254-4422-83e3-f812c5f459f5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:26.622959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:26.677472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:26.678511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.679146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.679631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.680613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:26.683279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:26.716755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 872/1750] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:26 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:26.720602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.731688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f5b82-a487-4722-b62f-a8d0e5af1e60 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:26.737198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:26.737807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.738081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.738351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.738992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:26.741021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:26.755488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 879/1751] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:26.758929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.775528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4e677-077c-42e7-ace3-506220b7c531 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:26.780948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b0bc2-8082-4da7-9b58-073ba3ded0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:26.781559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0bc2-8082-4da7-9b58-073ba3ded0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.781847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0bc2-8082-4da7-9b58-073ba3ded0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.782128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0bc2-8082-4da7-9b58-073ba3ded0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.782823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b0bc2-8082-4da7-9b58-073ba3ded0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:26.784474 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b0bc2-8082-4da7-9b58-073ba3ded0a7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:26.796130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 873/1752] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:26.800614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.811417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56913479-2c5f-47e5-a2ff-7849b47c3b25 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:26.818825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:26.819645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.819925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.820201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] PATH_INFO: `/v3/projects/4301ffc06fa64666bdc69ed6c3684bc0/users/f875d2d5fdb04b0ea98b38f716a9d179/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.844691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4301ffc06fa64666bdc69ed6c3684bc0, user_id=f875d2d5fdb04b0ea98b38f716a9d179, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:26.848990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:26.867416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 880/1753] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:26 2022] PUT /identity/v3/projects/4301ffc06fa64666bdc69ed6c3684bc0/users/f875d2d5fdb04b0ea98b38f716a9d179/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:26.878676 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e661-164b-4446-9ca4-af64e27b6f62 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:26.878960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e661-164b-4446-9ca4-af64e27b6f62 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:26.879233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e661-164b-4446-9ca4-af64e27b6f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:26.911801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b4b2e661-164b-4446-9ca4-af64e27b6f62 None None] MFA Rules not processed for user `f875d2d5fdb04b0ea98b38f716a9d179`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:26.913139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b4b2e661-164b-4446-9ca4-af64e27b6f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:26.939676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:26.941323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:27.027811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 874/1754] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:27.031828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e661-164b-4446-9ca4-af64e27b6f62 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:27.128545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e661-164b-4446-9ca4-af64e27b6f62 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.131693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cc042-ac1d-4237-82bc-f3957fb1db51 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.137711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.138449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.138782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.139060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.139858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.140340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.141230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.141911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.142989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] PATH_INFO: `/v3/users/162c210af61b4793b110700bb6cf84a3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.144994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=162c210af61b4793b110700bb6cf84a3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.148367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.149547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.187791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 875/1755] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:27 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:27.203929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:27.206086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:27.234524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162c210af61b4793b110700bb6cf84a3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:27.249825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162c210af61b4793b110700bb6cf84a3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:27.267874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162c210af61b4793b110700bb6cf84a3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:27.268613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162c210af61b4793b110700bb6cf84a3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:27.285964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:27.305071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5d080289-62c5-4919-981c-36502f5fd6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162c210af61b4793b110700bb6cf84a3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:27.312301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 881/1756] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:26 2022] DELETE /identity/v3/users/162c210af61b4793b110700bb6cf84a3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:27.322219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.322513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.322863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.345871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:27.347099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:27.365703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd2f85-3863-426e-8baf-bc8c09454504 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.439378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 882/1757] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:27.446871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.447488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.447763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.448033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.448787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.449356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:27.451026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.451147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:27.474033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 876/1758] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:27 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:27.478256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:27.481498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:27.494482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7157bf-654b-40d4-bc33-b15d4e5976ec admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.503621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.504794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.505083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.505499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.506853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.509873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.511802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69008d-978b-4369-9b05-a8904a315aa4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.518109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.518932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.519367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.519795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.524288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 877/1759] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:27.534990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.536144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.544313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:27.559442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd687-f2f8-4478-8336-08ad1b4a5944 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.562296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 883/1760] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:56:27.574230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.574905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.575184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.575457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] PATH_INFO: `/v3/projects/ad8af8106f134202bdbfc7efc77b1b21/users/b855aac4d4e24e45b9daa064fbd20dc6/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.599156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8af8106f134202bdbfc7efc77b1b21, user_id=b855aac4d4e24e45b9daa064fbd20dc6, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.602620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.620589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 878/1761] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:27 2022] PUT /identity/v3/projects/ad8af8106f134202bdbfc7efc77b1b21/users/b855aac4d4e24e45b9daa064fbd20dc6/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:27.625771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:27.638212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886321c9-6c0f-4027-8882-754bf9b1a3ae service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.709308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da963b-3d06-4be0-8aa8-69dea7fa3a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.710191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da963b-3d06-4be0-8aa8-69dea7fa3a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.710595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da963b-3d06-4be0-8aa8-69dea7fa3a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.710951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da963b-3d06-4be0-8aa8-69dea7fa3a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.711758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da963b-3d06-4be0-8aa8-69dea7fa3a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.713563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da963b-3d06-4be0-8aa8-69dea7fa3a2b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.715478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:27.726979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 884/1762] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:27.731154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:27.741726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb6843-5293-4a8c-89a1-db043a6b72b7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:27.748013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:27.752503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.752503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.752503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] PATH_INFO: `/v3/projects/ad8af8106f134202bdbfc7efc77b1b21/users/b855aac4d4e24e45b9daa064fbd20dc6/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.754913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8af8106f134202bdbfc7efc77b1b21, user_id=b855aac4d4e24e45b9daa064fbd20dc6, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:27.758419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:27.809482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 879/1763] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:27 2022] PUT /identity/v3/projects/ad8af8106f134202bdbfc7efc77b1b21/users/b855aac4d4e24e45b9daa064fbd20dc6/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:27.811476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:27.812306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:27.812776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:27.865295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] MFA Rules not processed for user `b855aac4d4e24e45b9daa064fbd20dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:27.867111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:28.007523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 885/1764] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:28.015413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:28.017625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:28.096518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2a89b-20cc-4834-a981-c45cf9bc3cf0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:28.100503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:28.101310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:28.101665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:28.102015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:28.108290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:28.109689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:28.112536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:28.114283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:28.124928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 880/1765] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:28.191922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d69f7f-41df-4c95-a4d6-9c0da1e29dc0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:28.198004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:28.198704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:28.199040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:28.199263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] PATH_INFO: `/v3/projects/a3a4ca5a7c7a42eb98e02132564f0d11` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:28.200489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3a4ca5a7c7a42eb98e02132564f0d11)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:28.202762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:28.257949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:28.267628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb24b34-a10d-49a5-bc55-026eae2ab085 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:28.271850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:28.294240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3a4ca5a7c7a42eb98e02132564f0d11', 'request_id': 'req-07758db3-d757-4557-aa89-292194b2cd58'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.304806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3a4ca5a7c7a42eb98e02132564f0d11', 'request_id': 'req-07758db3-d757-4557-aa89-292194b2cd58'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.312476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3a4ca5a7c7a42eb98e02132564f0d11', 'request_id': 'req-07758db3-d757-4557-aa89-292194b2cd58'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.330856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] The token cache is being invalidate because project a3a4ca5a7c7a42eb98e02132564f0d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:28.331297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.332353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.333993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 886/1766] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:28 2022] DELETE /identity/v3/projects/a3a4ca5a7c7a42eb98e02132564f0d11 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:28.337816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:28.339723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:28.356084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:28.368336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:28.369048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:28.369384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:28.369715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:28.371602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:28.430836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07758db3-d757-4557-aa89-292194b2cd58 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:28.446634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:28.448002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:28.448457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:28.448875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] PATH_INFO: `/v3/users/26f5347868f94f2f901bf1ef0526286e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:28.450228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f5347868f94f2f901bf1ef0526286e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:28.453596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:28.465809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:28.467271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:28.480218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 881/1767] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:56:28.513949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f5347868f94f2f901bf1ef0526286e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.522405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f5347868f94f2f901bf1ef0526286e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.530674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f5347868f94f2f901bf1ef0526286e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.531431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f5347868f94f2f901bf1ef0526286e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.550353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5c7bbf0f-87fb-4b3e-81f7-4759fe7829f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f5347868f94f2f901bf1ef0526286e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:28.559851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 887/1768] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:28 2022] DELETE /identity/v3/users/26f5347868f94f2f901bf1ef0526286e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:28.883278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:28.901319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045adf2b-b069-4a0f-b786-94d5423bc2b3 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:28.962461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:28.963105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:28.963391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:28.963669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] PATH_INFO: `/v3/projects/d9464bfe3a214a26b6abf04da53759cc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:28.965485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9464bfe3a214a26b6abf04da53759cc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:28.967398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.017900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9464bfe3a214a26b6abf04da53759cc', 'request_id': 'req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:29.025178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9464bfe3a214a26b6abf04da53759cc', 'request_id': 'req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:29.030105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:29.031341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9464bfe3a214a26b6abf04da53759cc', 'request_id': 'req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:29.044480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] The token cache is being invalidate because project d9464bfe3a214a26b6abf04da53759cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:29.044759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:29.045386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-05feaf71-7234-4f6b-ba5f-b5b49bfc37ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:29.047126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 882/1769] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:28 2022] DELETE /identity/v3/projects/d9464bfe3a214a26b6abf04da53759cc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:29.051789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fd42f-3b9d-4a87-85d9-f33ae6b4c492 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.052078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fd42f-3b9d-4a87-85d9-f33ae6b4c492 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.052390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fd42f-3b9d-4a87-85d9-f33ae6b4c492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.072422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2c6fd42f-3b9d-4a87-85d9-f33ae6b4c492 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:29.073722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2c6fd42f-3b9d-4a87-85d9-f33ae6b4c492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:29.139821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 888/1770] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:29.148177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ca913-4f66-4cff-9030-558dd3d570ff None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.148514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ca913-4f66-4cff-9030-558dd3d570ff None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.148789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ca913-4f66-4cff-9030-558dd3d570ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.171477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-957ca913-4f66-4cff-9030-558dd3d570ff None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:29.172760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-957ca913-4f66-4cff-9030-558dd3d570ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:29.184085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 883/1771] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:29.185351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fd42f-3b9d-4a87-85d9-f33ae6b4c492 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.196161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fd42f-3b9d-4a87-85d9-f33ae6b4c492 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.201149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.201831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.202116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.202386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.203041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:56:29.203340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.205524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.223772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 889/1772] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:56:29.225201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ca913-4f66-4cff-9030-558dd3d570ff None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.242890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ca913-4f66-4cff-9030-558dd3d570ff None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.258236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.258421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.258847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.259127 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.259774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.261966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.290600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 884/1773] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:29 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:29.294575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.305305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0fbe4-d295-4459-bb93-a6a64bf43fb9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.318818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:29.369735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.370350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.370659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.370934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.371566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.373521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.397907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 890/1774] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:29 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:29.399140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.410140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b144c-85fb-4d00-9818-d6166b019bc7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.416594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.417270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.417547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.417813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.418496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.420159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.432901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 885/1775] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:29.436666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.449204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d8639-44e0-4a92-baaa-abcca68d304f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.453935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ba2a7-c2b3-4895-8919-2ca9117cce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.454570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba2a7-c2b3-4895-8919-2ca9117cce0c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.454958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba2a7-c2b3-4895-8919-2ca9117cce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.455235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba2a7-c2b3-4895-8919-2ca9117cce0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.455964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ba2a7-c2b3-4895-8919-2ca9117cce0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.457624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ba2a7-c2b3-4895-8919-2ca9117cce0c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.468203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 891/1776] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:29.472175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.482464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b2f26-6607-4a3a-a584-c79b613d8fc7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.487642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.488316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.488597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.488873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] PATH_INFO: `/v3/projects/ff55a6432cac4fd18e53fee85eb77632/users/ccb042b16c2749a798c2de6eb7e8e347/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.504746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff55a6432cac4fd18e53fee85eb77632, user_id=ccb042b16c2749a798c2de6eb7e8e347, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.507512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.522538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 886/1777] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:29 2022] PUT /identity/v3/projects/ff55a6432cac4fd18e53fee85eb77632/users/ccb042b16c2749a798c2de6eb7e8e347/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:29.534147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ec04f-77b2-43c0-ba84-02421dd490b7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.534493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ec04f-77b2-43c0-ba84-02421dd490b7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.534775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ec04f-77b2-43c0-ba84-02421dd490b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.569088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5b3ec04f-77b2-43c0-ba84-02421dd490b7 None None] MFA Rules not processed for user `ccb042b16c2749a798c2de6eb7e8e347`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:29.570483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5b3ec04f-77b2-43c0-ba84-02421dd490b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:29.682851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 892/1778] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:29.687393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.772133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d178-3372-4154-bf41-ef7d6489abca admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.777680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.778390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.778779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.779126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.779857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.782183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.814079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 887/1779] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:29 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:29.815314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ec04f-77b2-43c0-ba84-02421dd490b7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.825904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ec04f-77b2-43c0-ba84-02421dd490b7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.885020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.885637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.885917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.886194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.886843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.888880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.912611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 893/1780] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:29 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:29.916909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.929085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79b487-57ed-4227-b5e3-9e6f1ae82190 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.934967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.935589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.935875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.936154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.936848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.939932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:29.949662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 888/1781] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:29.953914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:29.963908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6386dd1-7acf-4c52-97c5-6697c05a9826 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:29.969438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:29.970605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:29.970881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:29.971152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] PATH_INFO: `/v3/projects/49d2b9384aae4621adeb1aff2bc79222/users/e1d7953f62a8464489884d88d5266dd3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:29.983459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:29.987334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d2b9384aae4621adeb1aff2bc79222, user_id=e1d7953f62a8464489884d88d5266dd3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:29.990369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:30.008330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 894/1782] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:29 2022] PUT /identity/v3/projects/49d2b9384aae4621adeb1aff2bc79222/users/e1d7953f62a8464489884d88d5266dd3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:30.008503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:30.020838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c27073-dea8-4254-8fa5-571a7d3b20dc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:30.112607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825e114-e2d5-444c-a73a-254d3eb67c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:30.113466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e114-e2d5-444c-a73a-254d3eb67c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:30.113740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e114-e2d5-444c-a73a-254d3eb67c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:30.114014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e114-e2d5-444c-a73a-254d3eb67c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:30.114779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825e114-e2d5-444c-a73a-254d3eb67c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:30.116395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825e114-e2d5-444c-a73a-254d3eb67c22 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:30.119261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:30.134431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 889/1783] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:30.135726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:30.149036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09901293-e356-4331-974e-d1e09c02740a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:30.166273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:30.178862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:30.178862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:30.178862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] PATH_INFO: `/v3/projects/49d2b9384aae4621adeb1aff2bc79222/users/e1d7953f62a8464489884d88d5266dd3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:30.178862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d2b9384aae4621adeb1aff2bc79222, user_id=e1d7953f62a8464489884d88d5266dd3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:30.178862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:30.185107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 895/1784] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:30 2022] PUT /identity/v3/projects/49d2b9384aae4621adeb1aff2bc79222/users/e1d7953f62a8464489884d88d5266dd3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:30.195324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1350e-f7b5-49e3-ba91-1831d2a8701d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:30.195614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1350e-f7b5-49e3-ba91-1831d2a8701d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:30.195892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1350e-f7b5-49e3-ba91-1831d2a8701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:30.230294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2fe1350e-f7b5-49e3-ba91-1831d2a8701d None None] MFA Rules not processed for user `e1d7953f62a8464489884d88d5266dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:30.231630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2fe1350e-f7b5-49e3-ba91-1831d2a8701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:30.330180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 890/1785] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:30.343033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:30.346103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:30.427280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bf6a7-59d2-4d1a-b153-dfe2ebe0717d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:30.431737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:30.432464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:30.432808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:30.433146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:30.445776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:30.447096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:30.466532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 896/1786] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:56:30.545972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1350e-f7b5-49e3-ba91-1831d2a8701d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:30.555378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1350e-f7b5-49e3-ba91-1831d2a8701d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:30.558979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879de229-98f5-48f4-bdef-c31a59901ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:30.559616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879de229-98f5-48f4-bdef-c31a59901ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:30.559898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879de229-98f5-48f4-bdef-c31a59901ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:30.560172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879de229-98f5-48f4-bdef-c31a59901ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:30.618384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879de229-98f5-48f4-bdef-c31a59901ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:30.619725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879de229-98f5-48f4-bdef-c31a59901ad9 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:30.629493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 891/1787] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:31.204297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:31.206296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:31.297176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8dda0-eb57-4570-8a1f-41a23d700430 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:31.302089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:31.302729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:31.303009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:31.303282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] PATH_INFO: `/v3/users/f875d2d5fdb04b0ea98b38f716a9d179` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:31.304480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f875d2d5fdb04b0ea98b38f716a9d179)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:31.306403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:31.365310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f875d2d5fdb04b0ea98b38f716a9d179'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:31.374947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f875d2d5fdb04b0ea98b38f716a9d179'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:31.385315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f875d2d5fdb04b0ea98b38f716a9d179'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:31.385603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f875d2d5fdb04b0ea98b38f716a9d179'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:31.399822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f875d2d5fdb04b0ea98b38f716a9d179'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:31.406387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 897/1788] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:31 2022] DELETE /identity/v3/users/f875d2d5fdb04b0ea98b38f716a9d179 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:31.408482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:31.412193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:31.412716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:31.412799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:31.432673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:31.433993 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:31.506360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 892/1789] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:31.513175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:31.514603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:31.593498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81908c9f-883e-4a25-ad29-740e8cfce4eb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:31.600154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:31.601159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:31.601538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:31.601909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:31.614153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:31.616157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:31.634306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 898/1790] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:31.928306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:31.930430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:31.947624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967a115-e349-4cc9-9081-0f625eeb4548 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:31.956885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:31.957612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:31.957787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:31.958055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] PATH_INFO: `/v3/projects/4301ffc06fa64666bdc69ed6c3684bc0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:31.959279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4301ffc06fa64666bdc69ed6c3684bc0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:31.961776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:31.980400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:32.029864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4301ffc06fa64666bdc69ed6c3684bc0', 'request_id': 'req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.039440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4301ffc06fa64666bdc69ed6c3684bc0', 'request_id': 'req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.048434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4301ffc06fa64666bdc69ed6c3684bc0', 'request_id': 'req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.062612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] The token cache is being invalidate because project 4301ffc06fa64666bdc69ed6c3684bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:32.063064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.064351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.066188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 893/1791] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:31 2022] DELETE /identity/v3/projects/4301ffc06fa64666bdc69ed6c3684bc0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:32.070218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:32.085887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de2b42-9f1b-47a0-a926-8cc4571a33bc service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:32.202323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:32.203223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:32.203573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:32.203986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] PATH_INFO: `/v3/users/b855aac4d4e24e45b9daa064fbd20dc6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:32.205501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b855aac4d4e24e45b9daa064fbd20dc6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:32.207833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:32.281649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b855aac4d4e24e45b9daa064fbd20dc6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.297517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b855aac4d4e24e45b9daa064fbd20dc6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.308613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b855aac4d4e24e45b9daa064fbd20dc6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.309386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b855aac4d4e24e45b9daa064fbd20dc6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.322930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7dabfcae-082b-485b-be66-d93d4522c876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b855aac4d4e24e45b9daa064fbd20dc6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:32.329915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 899/1792] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:32 2022] DELETE /identity/v3/users/b855aac4d4e24e45b9daa064fbd20dc6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:33.080455 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:33.081912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:33.094659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd1b52-cb62-4a24-a815-7b6b9e2d02a8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:33.153374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:33.154006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.154282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.154554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] PATH_INFO: `/v3/projects/ad8af8106f134202bdbfc7efc77b1b21` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.155691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8af8106f134202bdbfc7efc77b1b21)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:33.157631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:33.209181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8af8106f134202bdbfc7efc77b1b21', 'request_id': 'req-4d7aac66-c487-47fa-898f-b036862f6e35'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.222057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8af8106f134202bdbfc7efc77b1b21', 'request_id': 'req-4d7aac66-c487-47fa-898f-b036862f6e35'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.228855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8af8106f134202bdbfc7efc77b1b21', 'request_id': 'req-4d7aac66-c487-47fa-898f-b036862f6e35'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.242107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] The token cache is being invalidate because project ad8af8106f134202bdbfc7efc77b1b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:33.242391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.243048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d7aac66-c487-47fa-898f-b036862f6e35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.244566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 894/1793] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:33 2022] DELETE /identity/v3/projects/ad8af8106f134202bdbfc7efc77b1b21 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:33.249357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741ce28-182f-4e78-be86-400699660997 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.249846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741ce28-182f-4e78-be86-400699660997 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.249943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741ce28-182f-4e78-be86-400699660997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.272178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9741ce28-182f-4e78-be86-400699660997 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:33.273418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9741ce28-182f-4e78-be86-400699660997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:33.309446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:33.372440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d2527-485c-4a44-9ad3-368c8cd8adb5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.372727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d2527-485c-4a44-9ad3-368c8cd8adb5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.373013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d2527-485c-4a44-9ad3-368c8cd8adb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.378146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 900/1794] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:33.466144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-181d2527-485c-4a44-9ad3-368c8cd8adb5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:33.467388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-181d2527-485c-4a44-9ad3-368c8cd8adb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:33.494294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 895/1795] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:33.498188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741ce28-182f-4e78-be86-400699660997 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:33.510276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741ce28-182f-4e78-be86-400699660997 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:33.516097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:33.516707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.517029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.517305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.518024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:56:33.518331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:33.520167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:33.535363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 901/1796] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:56:33.540570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d2527-485c-4a44-9ad3-368c8cd8adb5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:33.551972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d2527-485c-4a44-9ad3-368c8cd8adb5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:33.567273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:33.567902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.568182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.569036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.569160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:33.571136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:33.597910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:33.599440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:33.638878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af144f65-bef1-47bb-9f26-435eb90982be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:33.671806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:33.672507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.673043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.673357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] PATH_INFO: `/v3/users/ccb042b16c2749a798c2de6eb7e8e347` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.675558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb042b16c2749a798c2de6eb7e8e347)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:33.676943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 896/1797] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:33 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:33.678398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:33.681314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:33.709115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:33.791698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb042b16c2749a798c2de6eb7e8e347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.801113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb042b16c2749a798c2de6eb7e8e347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.810185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb042b16c2749a798c2de6eb7e8e347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.811122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb042b16c2749a798c2de6eb7e8e347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.832553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ee4de2b9-9bf5-470c-8bc6-922d5dbe85f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb042b16c2749a798c2de6eb7e8e347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:33.842978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 902/1798] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:33 2022] DELETE /identity/v3/users/ccb042b16c2749a798c2de6eb7e8e347 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:33.845881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380795b-ff79-4c23-8d54-fda33dac7f77 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:33.848114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.848450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.850482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.893123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:33.894341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:33.965378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:33.966146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:33.966430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:33.966702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:33.967347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:33.970162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:33.989660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 903/1799] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:34.000295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 897/1800] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:33 2022] POST /identity/v3/users => generated 291 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:34.005403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.005855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.007422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:34.008359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:34.022646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3f245-f877-4ffa-9c40-bbf9cdac2f23 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.028734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.029444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.029727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.030003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.030657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.032351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.043792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 898/1801] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:34.050229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.059973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:34.073076 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315623f-5800-4ef6-b6e7-7bd3cce6eab1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.087168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.087842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.090586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.090887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.092352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.096269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.149050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 899/1802] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:34.152328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d33d6-8760-4d21-b13d-9c872037a22c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.155084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.155213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf541b-5228-4cbd-8b14-e3aee580a304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.155801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf541b-5228-4cbd-8b14-e3aee580a304 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.156939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf541b-5228-4cbd-8b14-e3aee580a304 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.156939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf541b-5228-4cbd-8b14-e3aee580a304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.174784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf541b-5228-4cbd-8b14-e3aee580a304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.177293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf541b-5228-4cbd-8b14-e3aee580a304 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.177429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a584d-af92-4be8-9cae-d17add0e7a73 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.190683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.191367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.191646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.191919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] PATH_INFO: `/v3/projects/71f4dc8fe5dd4e128705d002ed1fa5dd/users/a8166c2800f24ee7a0d0986cc33c2c4b/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.193983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 904/1803] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:34.212878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f4dc8fe5dd4e128705d002ed1fa5dd, user_id=a8166c2800f24ee7a0d0986cc33c2c4b, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.216606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.233843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 900/1804] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:34 2022] PUT /identity/v3/projects/71f4dc8fe5dd4e128705d002ed1fa5dd/users/a8166c2800f24ee7a0d0986cc33c2c4b/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:34.247330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4518fa7-fd68-4077-9ba2-7e6832de3733 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.247578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4518fa7-fd68-4077-9ba2-7e6832de3733 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.247823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4518fa7-fd68-4077-9ba2-7e6832de3733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.284645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e4518fa7-fd68-4077-9ba2-7e6832de3733 None None] MFA Rules not processed for user `a8166c2800f24ee7a0d0986cc33c2c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:34.285863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e4518fa7-fd68-4077-9ba2-7e6832de3733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:34.514009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 905/1805] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:34.518974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.633906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:34.642375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c95c17-d9dd-430d-9b48-3e8f92ebb2ef admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.659715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.660669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.660951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.661226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.662114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.666381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.712498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 901/1806] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:34 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:34.712872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4518fa7-fd68-4077-9ba2-7e6832de3733 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.737027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4518fa7-fd68-4077-9ba2-7e6832de3733 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.806019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.807736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:34.897129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.897773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.898052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.898322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.898976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.901413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.927032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b99c57-c38c-4fa0-83fb-eb355a1c869c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.937805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.938547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.939036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.939436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] PATH_INFO: `/v3/projects/ff55a6432cac4fd18e53fee85eb77632` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.942618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 906/1807] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:34 2022] POST /identity/v3/users => generated 296 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:34.942764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff55a6432cac4fd18e53fee85eb77632)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.944042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.945908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.955711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c6faa-aa4c-469a-b1d1-686f8884fd7c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.960631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.961292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.961588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.961845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:34.962494 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:34.964186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:34.977339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 907/1808] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:34.981088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:34.993812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5fd4-159b-4380-b317-6f1acb68ae86 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:34.998684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:34.999338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:34.999630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:34.999892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] PATH_INFO: `/v3/projects/b6ef8af26f8f4085add4b45f4a1eaf6f/users/1d28b4a949674a57b9236a95308af6cc/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:35.017756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ef8af26f8f4085add4b45f4a1eaf6f, user_id=1d28b4a949674a57b9236a95308af6cc, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:35.020553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:35.020730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff55a6432cac4fd18e53fee85eb77632', 'request_id': 'req-f2a69648-d016-4604-ba7e-aa19dd94bb30'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.029051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff55a6432cac4fd18e53fee85eb77632', 'request_id': 'req-f2a69648-d016-4604-ba7e-aa19dd94bb30'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.036884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 908/1809] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:34 2022] PUT /identity/v3/projects/b6ef8af26f8f4085add4b45f4a1eaf6f/users/1d28b4a949674a57b9236a95308af6cc/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:35.038307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff55a6432cac4fd18e53fee85eb77632', 'request_id': 'req-f2a69648-d016-4604-ba7e-aa19dd94bb30'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.040760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:35.053541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:35.062577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] The token cache is being invalidate because project ff55a6432cac4fd18e53fee85eb77632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:35.063224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.063983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.066411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 902/1810] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:34 2022] DELETE /identity/v3/projects/ff55a6432cac4fd18e53fee85eb77632 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:35.070172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:35.071586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:35.119726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1f5f8-3a3e-4f97-b954-4a7a7c8afa8f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:35.125501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:35.127022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:35.127368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:35.127707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:35.128448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:35.130137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:35.140933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 909/1811] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:35.146939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:35.147065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a69648-d016-4604-ba7e-aa19dd94bb30 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:35.148885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:35.151545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:35.152269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:35.152624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:35.152985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] PATH_INFO: `/v3/users/e1d7953f62a8464489884d88d5266dd3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:35.154202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d7953f62a8464489884d88d5266dd3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:35.156199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:35.162490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf15ef-6e8c-411d-8d59-239b58863d91 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:35.167123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a8b1c-e056-43ba-a10a-dcff1290ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:35.167830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a8b1c-e056-43ba-a10a-dcff1290ce7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:35.168117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a8b1c-e056-43ba-a10a-dcff1290ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:35.168421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a8b1c-e056-43ba-a10a-dcff1290ce7c admin admin] PATH_INFO: `/v3/projects/b6ef8af26f8f4085add4b45f4a1eaf6f/users/1d28b4a949674a57b9236a95308af6cc/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:35.170326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a8b1c-e056-43ba-a10a-dcff1290ce7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ef8af26f8f4085add4b45f4a1eaf6f, user_id=1d28b4a949674a57b9236a95308af6cc, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:35.170807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:35.173053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a8b1c-e056-43ba-a10a-dcff1290ce7c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:35.186954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 910/1812] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:35 2022] PUT /identity/v3/projects/b6ef8af26f8f4085add4b45f4a1eaf6f/users/1d28b4a949674a57b9236a95308af6cc/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:35.199364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:35.199776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:35.200057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:35.217650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d7953f62a8464489884d88d5266dd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.230700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] MFA Rules not processed for user `1d28b4a949674a57b9236a95308af6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:35.231583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d7953f62a8464489884d88d5266dd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.232121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:35.239593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d7953f62a8464489884d88d5266dd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.240397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d7953f62a8464489884d88d5266dd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.241029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d7953f62a8464489884d88d5266dd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.256359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 903/1813] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:35 2022] DELETE /identity/v3/users/e1d7953f62a8464489884d88d5266dd3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:35.311397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 911/1814] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:35.324031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:35.325979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:35.398443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb5569-2782-4a16-95a9-32e7869df427 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:35.402642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73199e-cd26-46f8-b85f-f758f3fa4796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:35.403495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73199e-cd26-46f8-b85f-f758f3fa4796 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:35.403778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73199e-cd26-46f8-b85f-f758f3fa4796 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:35.404197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73199e-cd26-46f8-b85f-f758f3fa4796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:35.487719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73199e-cd26-46f8-b85f-f758f3fa4796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:35.489285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73199e-cd26-46f8-b85f-f758f3fa4796 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:35.502965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 904/1815] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:35.687374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:35.698546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:35.799987 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b5ad-ab52-4272-acd4-d5a935925ac7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:35.823900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:35.886391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:35.887035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:35.887552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:35.887829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] PATH_INFO: `/v3/projects/49d2b9384aae4621adeb1aff2bc79222` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:35.890012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d2b9384aae4621adeb1aff2bc79222)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:35.893303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:35.958010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d2b9384aae4621adeb1aff2bc79222', 'request_id': 'req-053441f5-c970-4c1a-b10a-4b51e70baccc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.970458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d2b9384aae4621adeb1aff2bc79222', 'request_id': 'req-053441f5-c970-4c1a-b10a-4b51e70baccc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.980082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d2b9384aae4621adeb1aff2bc79222', 'request_id': 'req-053441f5-c970-4c1a-b10a-4b51e70baccc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:35.998970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] The token cache is being invalidate because project 49d2b9384aae4621adeb1aff2bc79222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:35.999289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.000048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.005805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 912/1816] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:35 2022] DELETE /identity/v3/projects/49d2b9384aae4621adeb1aff2bc79222 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:36.007288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f6983-1147-4e79-950f-39deff242511 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.007577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f6983-1147-4e79-950f-39deff242511 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.007852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f6983-1147-4e79-950f-39deff242511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.029986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a96f6983-1147-4e79-950f-39deff242511 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:36.032601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a96f6983-1147-4e79-950f-39deff242511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.076364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.078143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.091475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:36.117767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 905/1817] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:36.125487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b711a-f0e6-47bd-9089-1cd148a60556 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.125856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b711a-f0e6-47bd-9089-1cd148a60556 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.127072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b711a-f0e6-47bd-9089-1cd148a60556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.151358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c92b711a-f0e6-47bd-9089-1cd148a60556 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:36.153078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c92b711a-f0e6-47bd-9089-1cd148a60556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.164015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 906/1818] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:36.167956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b711a-f0e6-47bd-9089-1cd148a60556 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.176483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053441f5-c970-4c1a-b10a-4b51e70baccc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.178979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b711a-f0e6-47bd-9089-1cd148a60556 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.181175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.181938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.182218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.182554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] PATH_INFO: `/v3/users/a8166c2800f24ee7a0d0986cc33c2c4b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.183829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8166c2800f24ee7a0d0986cc33c2c4b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.184408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.185132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.185517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.185782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.185964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.186709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:56:36.187118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.188876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.205491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 907/1819] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:56:36.206574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.217576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca2cbe-c098-49b4-9d83-2b1b73b7bc18 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.223821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.224594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.224834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.225192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.225790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.227904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.242167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8166c2800f24ee7a0d0986cc33c2c4b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.250603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8166c2800f24ee7a0d0986cc33c2c4b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.252791 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:36.259779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8166c2800f24ee7a0d0986cc33c2c4b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.260621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8166c2800f24ee7a0d0986cc33c2c4b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.262500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 908/1820] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:36.266378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.268678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.275297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0f067d2-e4b0-4b25-b36a-31e1485c7510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8166c2800f24ee7a0d0986cc33c2c4b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.282689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 913/1821] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:36 2022] DELETE /identity/v3/users/a8166c2800f24ee7a0d0986cc33c2c4b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:36.290600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.290969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.291315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.313435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:36.314632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.347478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d9490-4605-4c7c-b7e9-b6333524ab3a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.375642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 914/1822] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:36.385970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.387355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.411923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.412573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.412858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.413123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.413746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.415673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.438322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 909/1823] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:36.443715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.451386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf2cea-5927-45de-a2f0-1c54c85f2de8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.453290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d02e6-aa77-4ff7-83fa-f7d2d2f93df2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.455682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.456428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.456772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.457052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.458478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.459057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.459335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.459603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.460307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.462216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.466153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.467573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.473067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 910/1824] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:36.476828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.477216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 915/1825] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:56:36.486518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103bb14-3bc1-4fcb-900a-8b80190429b5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.491114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73520b7-c157-4ce4-a98a-e8bc89d74669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.491740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73520b7-c157-4ce4-a98a-e8bc89d74669 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.491963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73520b7-c157-4ce4-a98a-e8bc89d74669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.492362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73520b7-c157-4ce4-a98a-e8bc89d74669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.493107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73520b7-c157-4ce4-a98a-e8bc89d74669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.494698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73520b7-c157-4ce4-a98a-e8bc89d74669 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.505979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 911/1826] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:36.510723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.522086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bccbfc-9d03-4de9-80da-3c0a4c53a143 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.527556 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.528266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.528572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.528908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] PATH_INFO: `/v3/projects/8fae50d374f34e5586df9049c382cd9c/users/b35af1d149764970b5e4f80ca8097dcc/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.545655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fae50d374f34e5586df9049c382cd9c, user_id=b35af1d149764970b5e4f80ca8097dcc, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.548271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.561281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 916/1827] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:36 2022] PUT /identity/v3/projects/8fae50d374f34e5586df9049c382cd9c/users/b35af1d149764970b5e4f80ca8097dcc/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:36.573614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.573905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.574177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.601966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] MFA Rules not processed for user `b35af1d149764970b5e4f80ca8097dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:36.603174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.607124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:36.690628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 912/1828] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:36.695155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.756658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:36.767248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.768016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420123e0-9437-4116-a222-38d71d8dccff admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.768731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.777968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.779479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.779980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.780961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.781762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.784206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.784625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803c4d5-13a7-47ff-b98c-4b20eed2b2a1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.790653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.791523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.791949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.792450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] PATH_INFO: `/v3/projects/71f4dc8fe5dd4e128705d002ed1fa5dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.794718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f4dc8fe5dd4e128705d002ed1fa5dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.796916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.824571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 917/1829] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:36.828467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.854593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f4dc8fe5dd4e128705d002ed1fa5dd', 'request_id': 'req-fb2697bb-c961-4660-9204-978c6e180401'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.863446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f4dc8fe5dd4e128705d002ed1fa5dd', 'request_id': 'req-fb2697bb-c961-4660-9204-978c6e180401'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.869974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f4dc8fe5dd4e128705d002ed1fa5dd', 'request_id': 'req-fb2697bb-c961-4660-9204-978c6e180401'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.882869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] The token cache is being invalidate because project 71f4dc8fe5dd4e128705d002ed1fa5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:36.883331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.884158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:36.886663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 913/1830] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:36 2022] DELETE /identity/v3/projects/71f4dc8fe5dd4e128705d002ed1fa5dd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:36.892487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:36.896381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:36.935933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd5c4d-e228-442e-b44c-35d54ec2e4c2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.968026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2697bb-c961-4660-9204-978c6e180401 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:36.973583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.974212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.974520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.974792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] PATH_INFO: `/v3/users/1d28b4a949674a57b9236a95308af6cc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.976032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d28b4a949674a57b9236a95308af6cc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.977996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:36.993865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:36.994480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:36.994762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:36.995034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:36.995695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:36.997625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.015297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:37.023688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 918/1831] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:36 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:37.027763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.030032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.036100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d28b4a949674a57b9236a95308af6cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.044266 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d6c28-1f1b-4dab-a258-9093462b7dc4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.049540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.050275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.050914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.051151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d28b4a949674a57b9236a95308af6cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.051267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.052086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.053982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.058936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d28b4a949674a57b9236a95308af6cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.059702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d28b4a949674a57b9236a95308af6cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.060461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d28b4a949674a57b9236a95308af6cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.067341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 919/1832] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:37.071796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.073639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.078256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 914/1833] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:36 2022] DELETE /identity/v3/users/1d28b4a949674a57b9236a95308af6cc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:37.088690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583606ed-f4ba-4cd7-8e97-7c9caf11c928 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.174406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.175076 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.175352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.175624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] PATH_INFO: `/v3/projects/d1a54e3492804a3fba1a8ae222997600/users/a474c99acce3469e9fce52cac2698e7a/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.186014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:37.194873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a54e3492804a3fba1a8ae222997600, user_id=a474c99acce3469e9fce52cac2698e7a, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.197840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.211814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 920/1834] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:37 2022] PUT /identity/v3/projects/d1a54e3492804a3fba1a8ae222997600/users/a474c99acce3469e9fce52cac2698e7a/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:37.215897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.227343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1698-9e82-4f8a-81df-0bd6abd39980 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.287801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802e63-9bdc-481d-bd84-6c764d8e2f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.288513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802e63-9bdc-481d-bd84-6c764d8e2f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.288889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802e63-9bdc-481d-bd84-6c764d8e2f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.289105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802e63-9bdc-481d-bd84-6c764d8e2f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.289781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9802e63-9bdc-481d-bd84-6c764d8e2f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.291935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9802e63-9bdc-481d-bd84-6c764d8e2f25 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.311626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 915/1835] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:37.312577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.324775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51089-ee4c-4085-a443-ad0436adf184 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.330643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.331420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.331814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.332200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] PATH_INFO: `/v3/projects/d1a54e3492804a3fba1a8ae222997600/users/a474c99acce3469e9fce52cac2698e7a/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.334361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a54e3492804a3fba1a8ae222997600, user_id=a474c99acce3469e9fce52cac2698e7a, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.337326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.351844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 921/1836] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:37 2022] PUT /identity/v3/projects/d1a54e3492804a3fba1a8ae222997600/users/a474c99acce3469e9fce52cac2698e7a/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:37.362594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.362893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.363230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.396956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] MFA Rules not processed for user `a474c99acce3469e9fce52cac2698e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:37.398360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.441187 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.443431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.508606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 916/1837] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:37.521529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.522953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.523050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d540c2a-d65f-4a65-84a9-8c4948a998fd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.527861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.528518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.528793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.529063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] PATH_INFO: `/v3/projects/b6ef8af26f8f4085add4b45f4a1eaf6f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.530175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ef8af26f8f4085add4b45f4a1eaf6f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.532056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.555708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:37.581750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ef8af26f8f4085add4b45f4a1eaf6f', 'request_id': 'req-b80a297f-3b98-4aca-86cc-6e26ea2d760c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.582600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd652c-0fa0-4170-9b61-74a36a303f3e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.588005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ef8af26f8f4085add4b45f4a1eaf6f', 'request_id': 'req-b80a297f-3b98-4aca-86cc-6e26ea2d760c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.594127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ef8af26f8f4085add4b45f4a1eaf6f', 'request_id': 'req-b80a297f-3b98-4aca-86cc-6e26ea2d760c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.606220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] The token cache is being invalidate because project b6ef8af26f8f4085add4b45f4a1eaf6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:37.606515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.607152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b80a297f-3b98-4aca-86cc-6e26ea2d760c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:37.608714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 922/1838] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:37 2022] DELETE /identity/v3/projects/b6ef8af26f8f4085add4b45f4a1eaf6f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:37.614650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032abd-16d0-4f6a-bbec-6f3025876b63 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.614936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032abd-16d0-4f6a-bbec-6f3025876b63 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.615215 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032abd-16d0-4f6a-bbec-6f3025876b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.633394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.633996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.634378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.634659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.636392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.636499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c0032abd-16d0-4f6a-bbec-6f3025876b63 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:37.637697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c0032abd-16d0-4f6a-bbec-6f3025876b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.702807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 923/1839] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:37.709249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.709249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.711028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.711311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.711592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.715413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 917/1840] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:56:37.720150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:37.734843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:37.736712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.745260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 924/1841] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:37.749929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.760067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.761618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.762370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0197d-c610-4dd0-87d3-5c62d9ee5153 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.767207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.767831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.768136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.768515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.769188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:56:37.769513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.771197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.774299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2b0b-fcf0-4db2-9e78-1f5c2baa6b48 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.778419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.779018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.779295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.779565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.781779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:37.786696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 918/1842] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:56:37.787657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.797854 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d447228-9a3b-4459-b90f-8562a41061d7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.803036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.803619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.803897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.804174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.804832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.806699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.834879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 919/1843] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:37 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:37.838987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.842282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.843974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.855268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 925/1844] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:56:37.911559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cff7e-fea1-4756-9737-5db3a7c06642 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.925060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.925864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.926150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.926428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.927063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.929214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:37.939178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:37.962513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 920/1845] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:37 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:37.966556 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:37.988599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a50e6-6a9c-4e78-92b5-4e47806c325b service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:37.994053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:37.995235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:37.995773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:37.996057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:37.996931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:37.999204 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.018892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 926/1846] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:38.019683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.033589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead9ead-0898-44e4-99bb-91c58ef3c1f0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.047165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974ee5f-8c9d-4065-9809-1e40d161f5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.047875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974ee5f-8c9d-4065-9809-1e40d161f5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.048310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974ee5f-8c9d-4065-9809-1e40d161f5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.048679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974ee5f-8c9d-4065-9809-1e40d161f5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.049727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974ee5f-8c9d-4065-9809-1e40d161f5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.051537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974ee5f-8c9d-4065-9809-1e40d161f5d5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.067308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 921/1847] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:56:38.073451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.084355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d299-6b4f-40f1-ad8b-d2e546e0726f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.090177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.090777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.091057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.091334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] PATH_INFO: `/v3/projects/fbbc769400e542e5b03d50d825867063/users/19ae6b0e128e4306b34eb30d2b5bf4c5/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.107416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbc769400e542e5b03d50d825867063, user_id=19ae6b0e128e4306b34eb30d2b5bf4c5, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.110129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.132301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 927/1848] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:38 2022] PUT /identity/v3/projects/fbbc769400e542e5b03d50d825867063/users/19ae6b0e128e4306b34eb30d2b5bf4c5/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:38.138042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d4ad-9451-4584-81df-dfd0f8ac8702 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.138335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d4ad-9451-4584-81df-dfd0f8ac8702 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.138620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d4ad-9451-4584-81df-dfd0f8ac8702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.170837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-39d3d4ad-9451-4584-81df-dfd0f8ac8702 None None] MFA Rules not processed for user `19ae6b0e128e4306b34eb30d2b5bf4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:38.173100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-39d3d4ad-9451-4584-81df-dfd0f8ac8702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:38.244399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 922/1849] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:38.248935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.314148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708a81a-39c0-4f80-8c3a-e4b8dab7d099 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.318824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.319420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.319694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.319972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.320637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.322521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.353113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 928/1850] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:38 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:38.357094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3d4ad-9451-4584-81df-dfd0f8ac8702 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.368105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3d4ad-9451-4584-81df-dfd0f8ac8702 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.426593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.427184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.427463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.427736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.428476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.430377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.455531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 923/1851] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:38 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:56:38.456997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.468698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514af7a2-dda2-4c54-aef8-0dc969c14d71 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.474330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.474951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.475232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.475506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.476150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.477790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.490000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 929/1852] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:38.494349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.504962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2739de4-2a68-4e80-aece-71314a1243c4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.509604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.510258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.510538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.510811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] PATH_INFO: `/v3/projects/9ee0bddce3bf4592a64751e170a6c027/users/6bf1188113e447729e9ef0af1a471209/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.527733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee0bddce3bf4592a64751e170a6c027, user_id=6bf1188113e447729e9ef0af1a471209, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.530498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.547355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 924/1853] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:38 2022] PUT /identity/v3/projects/9ee0bddce3bf4592a64751e170a6c027/users/6bf1188113e447729e9ef0af1a471209/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:38.551291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.561385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394cfb1-7ff1-481c-bfba-cf8ca59e00c9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.592996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:38.617975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a90fee-17d2-499c-a1d8-fd3f970abdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.618504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a90fee-17d2-499c-a1d8-fd3f970abdac admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.618778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a90fee-17d2-499c-a1d8-fd3f970abdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.619045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a90fee-17d2-499c-a1d8-fd3f970abdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.619665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a90fee-17d2-499c-a1d8-fd3f970abdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.621288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a90fee-17d2-499c-a1d8-fd3f970abdac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.631396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 930/1854] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:38.635461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.646533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8204-cc4a-4d62-a346-0bd4a898f7de admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.651135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.651789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.652069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.652377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] PATH_INFO: `/v3/projects/9ee0bddce3bf4592a64751e170a6c027/users/6bf1188113e447729e9ef0af1a471209/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.654291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee0bddce3bf4592a64751e170a6c027, user_id=6bf1188113e447729e9ef0af1a471209, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.656962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.673024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 925/1855] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:38 2022] PUT /identity/v3/projects/9ee0bddce3bf4592a64751e170a6c027/users/6bf1188113e447729e9ef0af1a471209/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:38.678035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.678324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.678603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.716497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] MFA Rules not processed for user `6bf1188113e447729e9ef0af1a471209`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:38.717851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:38.790142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 931/1856] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:38.800686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:38.802339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:38.824888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:38.860797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061a70e-76cd-45bb-b531-20ceec5adec8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:38.864413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9028e-4dda-4b49-a811-0a9695bc0c89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:38.865007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9028e-4dda-4b49-a811-0a9695bc0c89 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:38.865281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9028e-4dda-4b49-a811-0a9695bc0c89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:38.865550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9028e-4dda-4b49-a811-0a9695bc0c89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:38.922487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9028e-4dda-4b49-a811-0a9695bc0c89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:38.923916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9028e-4dda-4b49-a811-0a9695bc0c89 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:38.932841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 926/1857] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:41.298059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:41.299869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:41.367073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcd967-4646-4f45-a800-696ecc51348e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:41.371680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:41.372560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:41.372761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:41.373028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] PATH_INFO: `/v3/users/b35af1d149764970b5e4f80ca8097dcc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:41.374166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35af1d149764970b5e4f80ca8097dcc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:41.376079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:41.425945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35af1d149764970b5e4f80ca8097dcc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:41.433684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35af1d149764970b5e4f80ca8097dcc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:41.436541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:41.448131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35af1d149764970b5e4f80ca8097dcc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:41.451745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35af1d149764970b5e4f80ca8097dcc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:41.489895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35af1d149764970b5e4f80ca8097dcc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:41.499332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 932/1858] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:41 2022] DELETE /identity/v3/users/b35af1d149764970b5e4f80ca8097dcc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:41.504090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:41.504649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:41.504649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:41.526519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:41.527856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:41.593308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 927/1859] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:41.610958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:41.612554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:41.673050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feba76b-4d42-4db0-be3f-d19e9cc26346 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:41.676946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:41.677551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:41.677823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:41.678090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:41.687199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:41.689236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:41.700387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 933/1860] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:41.993020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:41.994701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:42.009098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b5806-6756-4709-baf8-46192e13e50b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:42.014130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:42.014760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:42.015038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:42.015310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] PATH_INFO: `/v3/projects/8fae50d374f34e5586df9049c382cd9c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:42.017740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fae50d374f34e5586df9049c382cd9c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:42.019646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:42.087827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fae50d374f34e5586df9049c382cd9c', 'request_id': 'req-80fde655-ee4a-4c91-b21f-25071530afbf'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.096960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fae50d374f34e5586df9049c382cd9c', 'request_id': 'req-80fde655-ee4a-4c91-b21f-25071530afbf'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.104664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fae50d374f34e5586df9049c382cd9c', 'request_id': 'req-80fde655-ee4a-4c91-b21f-25071530afbf'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.112523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:42.118433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] The token cache is being invalidate because project 8fae50d374f34e5586df9049c382cd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:42.118981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.120605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.122679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 928/1861] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:41 2022] DELETE /identity/v3/projects/8fae50d374f34e5586df9049c382cd9c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:42.127216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:42.137152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d2f14-edf5-46b1-925e-78cc8d0d7376 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:42.225169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:42.226070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:42.226529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:42.226980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] PATH_INFO: `/v3/users/a474c99acce3469e9fce52cac2698e7a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:42.228442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a474c99acce3469e9fce52cac2698e7a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:42.230200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:42.294163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a474c99acce3469e9fce52cac2698e7a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.302646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a474c99acce3469e9fce52cac2698e7a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.310433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a474c99acce3469e9fce52cac2698e7a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.311171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a474c99acce3469e9fce52cac2698e7a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.323861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4b3785fb-3857-4ace-88b9-4e0ddfdb9f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a474c99acce3469e9fce52cac2698e7a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.325446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:42.331605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 934/1862] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:42 2022] DELETE /identity/v3/users/a474c99acce3469e9fce52cac2698e7a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:42.623154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:42.625101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:42.639837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fde655-ee4a-4c91-b21f-25071530afbf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:42.713396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:42.714038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:42.714319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:42.714591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] PATH_INFO: `/v3/projects/d1a54e3492804a3fba1a8ae222997600` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:42.715720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a54e3492804a3fba1a8ae222997600)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:42.718811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:42.772428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a54e3492804a3fba1a8ae222997600', 'request_id': 'req-0b36b522-4ac8-41e2-8648-3ca9d6102dab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.785181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a54e3492804a3fba1a8ae222997600', 'request_id': 'req-0b36b522-4ac8-41e2-8648-3ca9d6102dab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.792108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a54e3492804a3fba1a8ae222997600', 'request_id': 'req-0b36b522-4ac8-41e2-8648-3ca9d6102dab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.806145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] The token cache is being invalidate because project d1a54e3492804a3fba1a8ae222997600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:42.806432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.807101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b36b522-4ac8-41e2-8648-3ca9d6102dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:42.808676 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 929/1863] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:42 2022] DELETE /identity/v3/projects/d1a54e3492804a3fba1a8ae222997600 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:42.813732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f1844-57d2-47f5-946b-e79f489a64f5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:42.814111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f1844-57d2-47f5-946b-e79f489a64f5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:42.814600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f1844-57d2-47f5-946b-e79f489a64f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:42.854337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8c0f1844-57d2-47f5-946b-e79f489a64f5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:42.855570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8c0f1844-57d2-47f5-946b-e79f489a64f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:42.939873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 935/1864] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:42.947727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f53a75-0210-4837-82ca-d8013e5f1719 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:42.948015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f53a75-0210-4837-82ca-d8013e5f1719 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:42.948376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f53a75-0210-4837-82ca-d8013e5f1719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:42.973676 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b7f53a75-0210-4837-82ca-d8013e5f1719 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:42.975191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b7f53a75-0210-4837-82ca-d8013e5f1719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:42.988590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 930/1865] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:42.990139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f1844-57d2-47f5-946b-e79f489a64f5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.001026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f1844-57d2-47f5-946b-e79f489a64f5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.007148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.007932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.008281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.008570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.009228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:56:43.009529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.011150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.040700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 936/1866] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:56:43.046517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f53a75-0210-4837-82ca-d8013e5f1719 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.058600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f53a75-0210-4837-82ca-d8013e5f1719 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.064668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.065409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.065799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.066299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.067041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.069071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.109943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 931/1867] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:43 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:43.111094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.125157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1af9c-2698-47d1-b4a6-4a62ccb0cbf1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.192512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.193422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.193422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.193803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.194559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.196774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.234776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.244156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 937/1868] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:43 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:43.258666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34499ffe-2078-4c11-8524-6bad648800f8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.263568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.264196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.264654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.264944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.265716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.267393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.279617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 932/1869] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:43.284714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.305260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26b3f2-01d3-4931-8a37-33d22edb3421 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.311652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcca2b-57a3-4f74-97b0-d784f48f1236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.312884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcca2b-57a3-4f74-97b0-d784f48f1236 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.313171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcca2b-57a3-4f74-97b0-d784f48f1236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.313443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcca2b-57a3-4f74-97b0-d784f48f1236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.315285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcca2b-57a3-4f74-97b0-d784f48f1236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.317384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcca2b-57a3-4f74-97b0-d784f48f1236 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.335546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 938/1870] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:43.339382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.351887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3848c-4408-4b7b-95bc-592c97970efa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.357108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.357940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.358221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.358496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] PATH_INFO: `/v3/projects/fb5d2fc90f344f71bf1d8dc7548858dc/users/0b68982e394d4c6b9185e788da439dfd/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.374180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5d2fc90f344f71bf1d8dc7548858dc, user_id=0b68982e394d4c6b9185e788da439dfd, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.377141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.391505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 933/1871] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:43 2022] PUT /identity/v3/projects/fb5d2fc90f344f71bf1d8dc7548858dc/users/0b68982e394d4c6b9185e788da439dfd/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:43.402557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00fd7-0bae-4d24-9b66-c8e4e67d9e2e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.402843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00fd7-0bae-4d24-9b66-c8e4e67d9e2e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.403119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed00fd7-0bae-4d24-9b66-c8e4e67d9e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.434107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5ed00fd7-0bae-4d24-9b66-c8e4e67d9e2e None None] MFA Rules not processed for user `0b68982e394d4c6b9185e788da439dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:43.435529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5ed00fd7-0bae-4d24-9b66-c8e4e67d9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:43.499293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:43.511761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 939/1872] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:43.516414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.545773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:43.582766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71f1e-b97c-48dc-9fb2-eef08f30a8aa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.587514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.588121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.588431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.588703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.589341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.591251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.617807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 934/1873] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:43 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:43.621500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00fd7-0bae-4d24-9b66-c8e4e67d9e2e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.632437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed00fd7-0bae-4d24-9b66-c8e4e67d9e2e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.688842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.689455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.689730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.690027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.690653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.692627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.713213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 940/1874] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:43 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:43.717178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.728195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242b17c-7f61-413c-892c-d5fb328510ba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.732757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.733358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.733641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.733915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.734552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.736161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.746536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 935/1875] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:43.750495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.760573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf823738-53bc-426b-b683-6ed93b0e8556 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.765043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.765680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.765955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.766232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] PATH_INFO: `/v3/projects/96280fa70e604c0fa5f2bf29c67f8e67/users/20f158b370e94bc7a22d617be63a47e3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.780991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] RBAC: Authorizing `identity:create_grant(project_id=96280fa70e604c0fa5f2bf29c67f8e67, user_id=20f158b370e94bc7a22d617be63a47e3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.784267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.799709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 941/1876] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:43 2022] PUT /identity/v3/projects/96280fa70e604c0fa5f2bf29c67f8e67/users/20f158b370e94bc7a22d617be63a47e3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:43.803795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.813901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248e3d1-1048-4ce0-a93a-938e03abe8bf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.870355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dd4ab-ea2a-4964-aefe-8354fd246c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.870967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd4ab-ea2a-4964-aefe-8354fd246c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.871320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd4ab-ea2a-4964-aefe-8354fd246c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.871589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dd4ab-ea2a-4964-aefe-8354fd246c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.872370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dd4ab-ea2a-4964-aefe-8354fd246c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.873932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dd4ab-ea2a-4964-aefe-8354fd246c8d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.884344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 936/1877] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:43.888844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:43.899902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba233c-c402-4960-981b-c5d0b514237b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:43.904441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:43.905082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.905356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.905637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] PATH_INFO: `/v3/projects/96280fa70e604c0fa5f2bf29c67f8e67/users/20f158b370e94bc7a22d617be63a47e3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.907546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96280fa70e604c0fa5f2bf29c67f8e67, user_id=20f158b370e94bc7a22d617be63a47e3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:43.910275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:43.919996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 942/1878] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:43 2022] PUT /identity/v3/projects/96280fa70e604c0fa5f2bf29c67f8e67/users/20f158b370e94bc7a22d617be63a47e3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:43.930003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9e899-0f5d-41b0-bef2-4d683e1fe109 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:43.930294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9e899-0f5d-41b0-bef2-4d683e1fe109 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:43.930573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9e899-0f5d-41b0-bef2-4d683e1fe109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:43.958700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4ae9e899-0f5d-41b0-bef2-4d683e1fe109 None None] MFA Rules not processed for user `20f158b370e94bc7a22d617be63a47e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:43.959899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4ae9e899-0f5d-41b0-bef2-4d683e1fe109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:44.026138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 937/1879] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:44.037814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:44.039259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:44.097237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2276005-3af8-4e2b-ae96-b33e0e0e887a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:44.101329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:44.101931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:44.102205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:44.102473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:44.112036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:44.113516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:44.124053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 943/1880] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:44.213649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9e899-0f5d-41b0-bef2-4d683e1fe109 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:44.223240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9e899-0f5d-41b0-bef2-4d683e1fe109 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:44.226926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:44.227528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:44.227809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:44.228084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:44.245125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:44.285954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:44.287470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:44.296837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 938/1881] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:50.261056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:50.262491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:50.275033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55285-f641-4430-a9b7-f55cfb72fbec service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:50.279736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a4573-dfe6-4c7c-a39f-623ac4ff621d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:50.280482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4573-dfe6-4c7c-a39f-623ac4ff621d service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:50.280763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4573-dfe6-4c7c-a39f-623ac4ff621d service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:50.281038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4573-dfe6-4c7c-a39f-623ac4ff621d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:50.282811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-493a4573-dfe6-4c7c-a39f-623ac4ff621d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:50.334061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:50.377298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a4573-dfe6-4c7c-a39f-623ac4ff621d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:50.378676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a4573-dfe6-4c7c-a39f-623ac4ff621d service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:50.397339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 944/1882] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:50.909034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:50.971120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a71dd9-f338-48b6-9539-8f84f4b440c3 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:50.975778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:50.976441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:50.976722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:50.977000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] PATH_INFO: `/v3/users/0b68982e394d4c6b9185e788da439dfd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:50.978122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b68982e394d4c6b9185e788da439dfd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:50.980056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:51.025909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b68982e394d4c6b9185e788da439dfd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.037905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b68982e394d4c6b9185e788da439dfd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.044496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b68982e394d4c6b9185e788da439dfd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.045152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b68982e394d4c6b9185e788da439dfd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.045754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b68982e394d4c6b9185e788da439dfd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.058312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 939/1883] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:50 2022] DELETE /identity/v3/users/0b68982e394d4c6b9185e788da439dfd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:51.062922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:51.063207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:51.063506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:51.082393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:51.083554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:51.140045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 945/1884] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:51.150255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:51.151697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:51.194880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:51.211806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66f6c6-c6db-422f-a49b-237253fd5e04 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:51.215461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:51.216069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:51.216379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:51.216654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:51.226960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:51.228415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:51.237947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 940/1885] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:51.521508 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:51.522989 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:51.537461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2e5b-2a3a-4ba9-8e07-00389215641f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:51.542797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:51.543622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:51.544101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:51.544465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] PATH_INFO: `/v3/projects/fb5d2fc90f344f71bf1d8dc7548858dc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:51.545815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5d2fc90f344f71bf1d8dc7548858dc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:51.549832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:51.602744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5d2fc90f344f71bf1d8dc7548858dc', 'request_id': 'req-ebd26be3-a651-46c8-a79d-9405a593baf1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.608848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5d2fc90f344f71bf1d8dc7548858dc', 'request_id': 'req-ebd26be3-a651-46c8-a79d-9405a593baf1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.616469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5d2fc90f344f71bf1d8dc7548858dc', 'request_id': 'req-ebd26be3-a651-46c8-a79d-9405a593baf1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.628924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] The token cache is being invalidate because project fb5d2fc90f344f71bf1d8dc7548858dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:51.629214 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.629833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.631336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 946/1886] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:51 2022] DELETE /identity/v3/projects/fb5d2fc90f344f71bf1d8dc7548858dc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:56:51.635337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:51.645250 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c5ea-82af-4479-82f9-2952adef2b95 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:51.702246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:51.702795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:51.703072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:51.703347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] PATH_INFO: `/v3/users/20f158b370e94bc7a22d617be63a47e3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:51.704470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f158b370e94bc7a22d617be63a47e3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:51.706395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:51.752829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f158b370e94bc7a22d617be63a47e3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.764468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f158b370e94bc7a22d617be63a47e3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.772102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f158b370e94bc7a22d617be63a47e3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.772851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f158b370e94bc7a22d617be63a47e3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.773494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a1f99d7c-7660-4e17-bec6-d68e0a761f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f158b370e94bc7a22d617be63a47e3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:51.788180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 941/1887] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:51 2022] DELETE /identity/v3/users/20f158b370e94bc7a22d617be63a47e3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:52.088678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.090129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:52.103714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd26be3-a651-46c8-a79d-9405a593baf1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.159822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.160618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.160885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.161157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] PATH_INFO: `/v3/projects/96280fa70e604c0fa5f2bf29c67f8e67` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.162267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=96280fa70e604c0fa5f2bf29c67f8e67)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.164154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.171825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:52.216068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96280fa70e604c0fa5f2bf29c67f8e67', 'request_id': 'req-f7286a6a-6855-4127-b541-f16c78d418fa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:52.222140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96280fa70e604c0fa5f2bf29c67f8e67', 'request_id': 'req-f7286a6a-6855-4127-b541-f16c78d418fa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:52.227531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96280fa70e604c0fa5f2bf29c67f8e67', 'request_id': 'req-f7286a6a-6855-4127-b541-f16c78d418fa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:52.239618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] The token cache is being invalidate because project 96280fa70e604c0fa5f2bf29c67f8e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:56:52.239903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:52.240639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f7286a6a-6855-4127-b541-f16c78d418fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:52.242132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 947/1888] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:56:52 2022] DELETE /identity/v3/projects/96280fa70e604c0fa5f2bf29c67f8e67 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:52.246986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949b43d-9e72-493b-8498-8c7f29326e86 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.247448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949b43d-9e72-493b-8498-8c7f29326e86 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.247556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949b43d-9e72-493b-8498-8c7f29326e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.267988 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f949b43d-9e72-493b-8498-8c7f29326e86 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:52.269218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f949b43d-9e72-493b-8498-8c7f29326e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:52.318461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:52.328864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 942/1889] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:52.337734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8d7f5-107b-4783-aac8-5a99cec53cd4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.338022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8d7f5-107b-4783-aac8-5a99cec53cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.338299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8d7f5-107b-4783-aac8-5a99cec53cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.357818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5bf8d7f5-107b-4783-aac8-5a99cec53cd4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:52.358992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5bf8d7f5-107b-4783-aac8-5a99cec53cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:52.367299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 948/1890] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:56:52.371406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949b43d-9e72-493b-8498-8c7f29326e86 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.381446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949b43d-9e72-493b-8498-8c7f29326e86 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.385953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.386589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.386868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.387142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.387783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:56:52.388085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.389791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.401028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 943/1891] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:56:52.405378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8d7f5-107b-4783-aac8-5a99cec53cd4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.415479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8d7f5-107b-4783-aac8-5a99cec53cd4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.420336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.420942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.421236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.421503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.422162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.424696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.451504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 949/1892] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:56:52 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:52.455569 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.465394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08abfbf-0605-40fa-98ec-0c848b501260 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.525244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.533538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.533538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.533538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.533538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.533538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.552881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 944/1893] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:56:52 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:56:52.557564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.568430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984533e-e39c-41dc-b4c9-4bd1d97711c2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.574030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.574639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.574919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.575190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.575828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.577464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.589717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 950/1894] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:52.597551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.611611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94e698-2fe5-4c70-9017-87a983371c58 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.617999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fb1e5-feeb-4826-b57d-fd17ac48a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.618724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fb1e5-feeb-4826-b57d-fd17ac48a677 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.619063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fb1e5-feeb-4826-b57d-fd17ac48a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.619346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fb1e5-feeb-4826-b57d-fd17ac48a677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.620164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fb1e5-feeb-4826-b57d-fd17ac48a677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.621891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fb1e5-feeb-4826-b57d-fd17ac48a677 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.636301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 945/1895] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:56:52.638961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.653094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ec05-3f8e-42e8-b08c-867ba546bae1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.661228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.662667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.662973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.663254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] PATH_INFO: `/v3/projects/77999249e1924fb5a6346bb22ea17a78/users/b0954439d74a46ac914bcc7b669ac360/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.682163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77999249e1924fb5a6346bb22ea17a78, user_id=b0954439d74a46ac914bcc7b669ac360, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.684919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.699763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 951/1896] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:56:52 2022] PUT /identity/v3/projects/77999249e1924fb5a6346bb22ea17a78/users/b0954439d74a46ac914bcc7b669ac360/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:52.711149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.711571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.711803 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.742080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] MFA Rules not processed for user `b0954439d74a46ac914bcc7b669ac360`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:52.743590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:52.812089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 946/1897] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:52.827066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:52.829442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:52.895950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f98b4-8e8e-4888-bb27-75e597b2b5e8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:52.900211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62e82f-1e1b-4493-b166-a09ef90cbe87 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:52.900890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e82f-1e1b-4493-b166-a09ef90cbe87 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:52.901169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e82f-1e1b-4493-b166-a09ef90cbe87 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:52.901447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e82f-1e1b-4493-b166-a09ef90cbe87 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:52.914680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62e82f-1e1b-4493-b166-a09ef90cbe87 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:52.916905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62e82f-1e1b-4493-b166-a09ef90cbe87 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:52.933141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 952/1898] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:56:59.599114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:59.600542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:59.669258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:59.677092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14badf21-2b0d-4480-8713-d99ce150d0f7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:59.683602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:59.684476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:59.685119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:59.685575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] PATH_INFO: `/v3/users/19ae6b0e128e4306b34eb30d2b5bf4c5` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:59.687431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ae6b0e128e4306b34eb30d2b5bf4c5)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:59.690462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:59.745400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ae6b0e128e4306b34eb30d2b5bf4c5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:59.760196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ae6b0e128e4306b34eb30d2b5bf4c5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:59.769701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ae6b0e128e4306b34eb30d2b5bf4c5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:59.770452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ae6b0e128e4306b34eb30d2b5bf4c5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:59.771081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ae6b0e128e4306b34eb30d2b5bf4c5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:56:59.783678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 947/1899] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:56:59 2022] DELETE /identity/v3/users/19ae6b0e128e4306b34eb30d2b5bf4c5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:56:59.788083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:59.788631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:59.788747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:59.808423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:56:59.809832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:59.845696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:56:59.879889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 953/1900] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:56:59.889751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:56:59.893255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:56:59.955210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529253a8-d835-4d54-a01f-db62bceaa8b7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:56:59.958884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:56:59.959495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:56:59.959778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:56:59.960051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:56:59.969111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:56:59.970560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:56:59.980269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 948/1901] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:00.281174 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:00.283040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:00.296999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc58f50-c656-480c-b6df-62f35069ca89 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:00.309668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:00.310450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:00.310825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:00.311146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] PATH_INFO: `/v3/projects/fbbc769400e542e5b03d50d825867063` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:00.313183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbbc769400e542e5b03d50d825867063)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:00.316512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:00.379522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbbc769400e542e5b03d50d825867063', 'request_id': 'req-67cc093e-cdb6-435c-b563-f700cd6763fc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.385689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbbc769400e542e5b03d50d825867063', 'request_id': 'req-67cc093e-cdb6-435c-b563-f700cd6763fc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.391315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbbc769400e542e5b03d50d825867063', 'request_id': 'req-67cc093e-cdb6-435c-b563-f700cd6763fc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.404190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] The token cache is being invalidate because project fbbc769400e542e5b03d50d825867063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:00.404593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.405289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.406980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 954/1902] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:00 2022] DELETE /identity/v3/projects/fbbc769400e542e5b03d50d825867063 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:00.411391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:00.421143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6acd3-af59-46dc-b4f4-fd2c149118e6 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:00.489374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:00.490053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:00.490336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:00.490612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] PATH_INFO: `/v3/users/6bf1188113e447729e9ef0af1a471209` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:00.491742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bf1188113e447729e9ef0af1a471209)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:00.493719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:00.515452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:00.540808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bf1188113e447729e9ef0af1a471209'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.553254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bf1188113e447729e9ef0af1a471209'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.559944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bf1188113e447729e9ef0af1a471209'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.560614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bf1188113e447729e9ef0af1a471209'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.561239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-840bf84b-376d-4e41-96a2-1c7d81bb8f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bf1188113e447729e9ef0af1a471209'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.573689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 949/1903] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:00 2022] DELETE /identity/v3/users/6bf1188113e447729e9ef0af1a471209 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:00.853829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:00.855328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:00.870023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc093e-cdb6-435c-b563-f700cd6763fc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:00.939228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:00.939924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:00.940269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:00.940589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] PATH_INFO: `/v3/projects/9ee0bddce3bf4592a64751e170a6c027` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:00.941791 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee0bddce3bf4592a64751e170a6c027)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:00.943801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:00.991506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee0bddce3bf4592a64751e170a6c027', 'request_id': 'req-c541a3e9-69c9-4006-a2b4-dc6a58da779d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:00.997884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee0bddce3bf4592a64751e170a6c027', 'request_id': 'req-c541a3e9-69c9-4006-a2b4-dc6a58da779d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:01.002919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:01.004143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee0bddce3bf4592a64751e170a6c027', 'request_id': 'req-c541a3e9-69c9-4006-a2b4-dc6a58da779d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:01.018804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] The token cache is being invalidate because project 9ee0bddce3bf4592a64751e170a6c027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:01.019097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:01.019755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c541a3e9-69c9-4006-a2b4-dc6a58da779d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:01.021543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 955/1904] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:00 2022] DELETE /identity/v3/projects/9ee0bddce3bf4592a64751e170a6c027 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:01.030150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56fe7a-0a56-4ae4-b87d-05fa3b8d0559 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.030650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56fe7a-0a56-4ae4-b87d-05fa3b8d0559 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.030828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56fe7a-0a56-4ae4-b87d-05fa3b8d0559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.051121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ef56fe7a-0a56-4ae4-b87d-05fa3b8d0559 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:01.052406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef56fe7a-0a56-4ae4-b87d-05fa3b8d0559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:01.115466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 950/1905] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:01.124113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f575-dba7-4724-9ff3-993262ac9ee2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.124446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f575-dba7-4724-9ff3-993262ac9ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.124725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f575-dba7-4724-9ff3-993262ac9ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.145524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0756f575-dba7-4724-9ff3-993262ac9ee2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:01.146729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0756f575-dba7-4724-9ff3-993262ac9ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:01.154697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 956/1906] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:01.159022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56fe7a-0a56-4ae4-b87d-05fa3b8d0559 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:01.171409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56fe7a-0a56-4ae4-b87d-05fa3b8d0559 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:01.175946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:01.176566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.176844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.177113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.177750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:01.178045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:01.179606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:01.191494 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 951/1907] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:01.195635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756f575-dba7-4724-9ff3-993262ac9ee2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:01.205408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756f575-dba7-4724-9ff3-993262ac9ee2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:01.210079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:01.210673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.210950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.211221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.211859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:01.213792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:01.240021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 957/1908] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:01 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:01.243890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:01.254430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee39d9-ee64-40e6-a20c-85ab5cfb41c5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:01.310264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:01.310870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.311151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.311423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.312048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:01.314531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:01.335682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 952/1909] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:01 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:01.339736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:01.350466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2612b7f-8964-4c69-8581-2468b8118e79 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:01.356114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:01.356909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.357354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.357673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.358380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:01.360407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:01.371328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 958/1910] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:01.375174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:01.386298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc842dd5-7aff-4d3d-83f4-3cae9e94d1cb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:01.391016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76d7e5-533b-464c-8644-37d64db2fe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:01.391652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76d7e5-533b-464c-8644-37d64db2fe31 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.392639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76d7e5-533b-464c-8644-37d64db2fe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.392916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76d7e5-533b-464c-8644-37d64db2fe31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.393553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76d7e5-533b-464c-8644-37d64db2fe31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:01.398952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76d7e5-533b-464c-8644-37d64db2fe31 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:01.411408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 953/1911] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:01.415797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:01.425740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ab372-58dc-41f8-80f2-85ec1daa00d3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:01.430280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:01.430922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.431203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.431490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] PATH_INFO: `/v3/projects/7c116c9cfbc144769d453e218b02739b/users/1fafe6ce050545b8870f88d4ae0aa1ca/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.447052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c116c9cfbc144769d453e218b02739b, user_id=1fafe6ce050545b8870f88d4ae0aa1ca, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:01.449719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:01.462975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 959/1912] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:01 2022] PUT /identity/v3/projects/7c116c9cfbc144769d453e218b02739b/users/1fafe6ce050545b8870f88d4ae0aa1ca/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:01.467833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.468119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.468428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.475442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:01.489290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:01.490558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:01.551325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 954/1913] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:01.560885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:01.562526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:01.619479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ab151-fe67-4472-9fcb-15b43fe0bad9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:01.623149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df0d59-0022-40d7-be34-4e380743d50d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:01.623785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df0d59-0022-40d7-be34-4e380743d50d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:01.624065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df0d59-0022-40d7-be34-4e380743d50d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:01.624379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df0d59-0022-40d7-be34-4e380743d50d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:01.634690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df0d59-0022-40d7-be34-4e380743d50d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:01.636039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df0d59-0022-40d7-be34-4e380743d50d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:01.646433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 960/1914] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:02.526543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:02.528638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:02.541912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f45c4e-7c33-4df9-ba1c-0afb6b4bf03e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:02.546899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:02.547591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:02.547876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:02.549031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] PATH_INFO: `/v3/users/64614891540a4f79913d71672b081781` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:02.550472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64614891540a4f79913d71672b081781)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:02.552635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:02.604268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64614891540a4f79913d71672b081781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:02.618220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64614891540a4f79913d71672b081781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:02.625891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64614891540a4f79913d71672b081781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:02.627540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64614891540a4f79913d71672b081781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:02.628165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64614891540a4f79913d71672b081781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:02.644117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 955/1915] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:02 2022] DELETE /identity/v3/users/64614891540a4f79913d71672b081781 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:02.648474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:02.649319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:02.649686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:02.674448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:02.677990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:02.758924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 961/1916] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:02.777094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:02.778550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:02.842513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dca72-5a7b-4c67-8c83-5098cb883e66 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:02.848081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:02.848850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:02.850416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:02.850806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:02.873565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:02.879588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:02.907992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 956/1917] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:03.710612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:03.712322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:03.735262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e325951-fbc7-42e8-9fde-aec6f8885ff8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:03.742637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:03.743569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:03.744075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:03.744530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] PATH_INFO: `/v3/projects/80d31576169349449039f7766167b1f3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:03.745883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d31576169349449039f7766167b1f3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:03.748644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:03.778608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:03.815996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d31576169349449039f7766167b1f3', 'request_id': 'req-eecb07e4-c2d1-4538-8a3a-ec2aed538355'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:03.823809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d31576169349449039f7766167b1f3', 'request_id': 'req-eecb07e4-c2d1-4538-8a3a-ec2aed538355'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:03.839113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d31576169349449039f7766167b1f3', 'request_id': 'req-eecb07e4-c2d1-4538-8a3a-ec2aed538355'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:03.863156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] The token cache is being invalidate because project 80d31576169349449039f7766167b1f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:03.863525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:03.864343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:03.866241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 962/1918] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:03 2022] DELETE /identity/v3/projects/80d31576169349449039f7766167b1f3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:03.870010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:03.888235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a7464-26d6-47d4-be86-528184d4ed81 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:03.939344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:04.026811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:04.027464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:04.027741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:04.028012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] PATH_INFO: `/v3/users/d5ea918c23ad47bf98f147b5e0a386ff` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:04.030612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ea918c23ad47bf98f147b5e0a386ff)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:04.032616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:04.129820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ea918c23ad47bf98f147b5e0a386ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.145551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ea918c23ad47bf98f147b5e0a386ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.153506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ea918c23ad47bf98f147b5e0a386ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.154678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ea918c23ad47bf98f147b5e0a386ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.155528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60796f7b-b8c6-4aba-bd1d-0cbf5dcc3a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ea918c23ad47bf98f147b5e0a386ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.176020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 957/1919] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:03 2022] DELETE /identity/v3/users/d5ea918c23ad47bf98f147b5e0a386ff => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:04.553196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:04.554697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:04.582852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb07e4-c2d1-4538-8a3a-ec2aed538355 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:04.713137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:04.714110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:04.715339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:04.715691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] PATH_INFO: `/v3/projects/00f1862eb58f493c9f8a29bf77d52a0e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:04.718900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f1862eb58f493c9f8a29bf77d52a0e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:04.721747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:04.804955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f1862eb58f493c9f8a29bf77d52a0e', 'request_id': 'req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.813068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f1862eb58f493c9f8a29bf77d52a0e', 'request_id': 'req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.820015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f1862eb58f493c9f8a29bf77d52a0e', 'request_id': 'req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.873207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] The token cache is being invalidate because project 00f1862eb58f493c9f8a29bf77d52a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:04.873520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.874223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe9e240c-bc1c-48ef-9ef5-5d3e99164f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:04.880613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 963/1920] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:04 2022] DELETE /identity/v3/projects/00f1862eb58f493c9f8a29bf77d52a0e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:04.882424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2056b0-ca1a-4af1-98f7-904701ba14b6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:04.882901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2056b0-ca1a-4af1-98f7-904701ba14b6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:04.882995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2056b0-ca1a-4af1-98f7-904701ba14b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:04.914935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7d2056b0-ca1a-4af1-98f7-904701ba14b6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:04.918226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7d2056b0-ca1a-4af1-98f7-904701ba14b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:05.057118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 958/1921] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:05.065640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a66a1-c604-418b-b3cc-b4a2cf8cac64 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.066164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a66a1-c604-418b-b3cc-b4a2cf8cac64 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.066471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a66a1-c604-418b-b3cc-b4a2cf8cac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.123707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-731a66a1-c604-418b-b3cc-b4a2cf8cac64 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:05.128959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-731a66a1-c604-418b-b3cc-b4a2cf8cac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:05.150781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 964/1922] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:05.154985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2056b0-ca1a-4af1-98f7-904701ba14b6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.169967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2056b0-ca1a-4af1-98f7-904701ba14b6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:05.181775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:05.182408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.182694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.182973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.183635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:05.183941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:05.190505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:05.219712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 959/1923] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:05.224980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a66a1-c604-418b-b3cc-b4a2cf8cac64 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.244136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a66a1-c604-418b-b3cc-b4a2cf8cac64 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:05.264193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:05.264866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.265139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.265413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.266060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:05.268584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:05.327161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 965/1924] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:05 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:05.328354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.346762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c8385-9771-45e0-ac6c-86e79f395bc9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:05.379415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:05.426824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:05.427345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.427560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.427839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.428515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:05.430650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:05.464582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 960/1925] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:05 2022] POST /identity/v3/users => generated 282 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:05.465963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.488312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc2f8f-28d4-4994-9658-a37a6faeada5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:05.495011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:05.496083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.496575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.497035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.497765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:05.499380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:05.511329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 966/1926] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:05.515399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.526172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5900e1-c7e3-4427-b191-b67e0af82cc7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:05.536890 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02797caf-3275-4d01-8a52-edf1156c82c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:05.537920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02797caf-3275-4d01-8a52-edf1156c82c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.538204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02797caf-3275-4d01-8a52-edf1156c82c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.538480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02797caf-3275-4d01-8a52-edf1156c82c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.539162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02797caf-3275-4d01-8a52-edf1156c82c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:05.540803 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02797caf-3275-4d01-8a52-edf1156c82c8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:05.557213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 961/1927] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:05.561357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.568072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:05.579377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0b919-7a11-4e90-bb03-5d081b9f04ac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:05.586298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:05.587143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.587444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.587837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] PATH_INFO: `/v3/projects/2d07fd52c5184080b044da1f1ced2f2a/users/1d0c8a9a96a048529b9b0cb4a68bc348/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.605596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d07fd52c5184080b044da1f1ced2f2a, user_id=1d0c8a9a96a048529b9b0cb4a68bc348, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:05.611064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:05.642425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 967/1928] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:05 2022] PUT /identity/v3/projects/2d07fd52c5184080b044da1f1ced2f2a/users/1d0c8a9a96a048529b9b0cb4a68bc348/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:05.654723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e3961-76bd-478f-bc51-1d6b5c132c4a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.655108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e3961-76bd-478f-bc51-1d6b5c132c4a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.655347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e3961-76bd-478f-bc51-1d6b5c132c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.698388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-408e3961-76bd-478f-bc51-1d6b5c132c4a None None] MFA Rules not processed for user `1d0c8a9a96a048529b9b0cb4a68bc348`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:05.699636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-408e3961-76bd-478f-bc51-1d6b5c132c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:05.784034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 962/1929] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:05.789615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.885890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a4eba-75d6-4317-9097-ab675f09a0cb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:05.891520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:05.892928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:05.893427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:05.893877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:05.894748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:05.896906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:05.934292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 968/1930] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:05 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:05.938138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e3961-76bd-478f-bc51-1d6b5c132c4a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:05.955758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e3961-76bd-478f-bc51-1d6b5c132c4a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:06.052079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:06.053220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.053822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.054393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.055327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:06.057777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:06.090216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 963/1931] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:05 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:06.091613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:06.103648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aba83-379d-4991-bb43-5b5487876e7a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:06.109057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:06.109668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.109941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.110210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.110939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:06.112565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:06.123838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 969/1932] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:06.128369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:06.138640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e11da-fcec-4e54-867e-ec8fb7c30bed admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:06.144100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:06.145069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.145351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.145632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] PATH_INFO: `/v3/projects/7184db17d1714ace8151b5a7d917a4f0/users/fdfadf6312cb4fbbad5555689323fe81/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.181088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7184db17d1714ace8151b5a7d917a4f0, user_id=fdfadf6312cb4fbbad5555689323fe81, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:06.190921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:06.206126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:06.220637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 964/1933] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:06 2022] PUT /identity/v3/projects/7184db17d1714ace8151b5a7d917a4f0/users/fdfadf6312cb4fbbad5555689323fe81/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:06.222065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:06.233573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe05d6-c57a-46d7-9f06-686ad676b386 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:06.307308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e3bcf-64a3-4265-8e11-d4b928faa4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:06.308010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e3bcf-64a3-4265-8e11-d4b928faa4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.308356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e3bcf-64a3-4265-8e11-d4b928faa4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.308655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e3bcf-64a3-4265-8e11-d4b928faa4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.309345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e3bcf-64a3-4265-8e11-d4b928faa4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:06.310965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e3bcf-64a3-4265-8e11-d4b928faa4e0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:06.325112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 970/1934] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:06.327292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:06.338789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da294594-f34a-498a-927d-05326b0e006f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:06.344598 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f74d0-aa28-490e-907c-7c86eeee0390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:06.346071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f74d0-aa28-490e-907c-7c86eeee0390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.346209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f74d0-aa28-490e-907c-7c86eeee0390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.346404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f74d0-aa28-490e-907c-7c86eeee0390 admin admin] PATH_INFO: `/v3/projects/7184db17d1714ace8151b5a7d917a4f0/users/fdfadf6312cb4fbbad5555689323fe81/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.348479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f74d0-aa28-490e-907c-7c86eeee0390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7184db17d1714ace8151b5a7d917a4f0, user_id=fdfadf6312cb4fbbad5555689323fe81, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:06.351172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f74d0-aa28-490e-907c-7c86eeee0390 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:06.372329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 965/1935] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:06 2022] PUT /identity/v3/projects/7184db17d1714ace8151b5a7d917a4f0/users/fdfadf6312cb4fbbad5555689323fe81/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:06.379813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.380582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.381009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.417071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] MFA Rules not processed for user `fdfadf6312cb4fbbad5555689323fe81`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:06.418284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:06.480032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:06.565510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.566923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.569376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.585493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 971/1936] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:06.592403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:06.595234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:06.619254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] MFA Rules not processed for user `1fafe6ce050545b8870f88d4ae0aa1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:06.620508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:06.686904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1de2d-b81e-4294-a20c-1cfab42cda03 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:06.691520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:06.692537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.693097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.693570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.708132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:06.710547 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:06.730863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 966/1937] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:06.736653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 972/1938] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:06.737327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:06.739443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:06.878875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93950a78-b5a6-4188-a3ae-3d5f87981ed8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:06.884057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:06.884824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:06.885196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:06.885589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:06.886686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:06.889233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:06.954588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 967/1939] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:06 2022] POST /identity/v3/projects => generated 371 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:06.958970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:06.978218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c9485-176f-442d-918d-bcd8e7ee04f9 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:07.090817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:07.091860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.092480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.092903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.093705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:07.096325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:07.141900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 973/1940] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:06 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:07.145656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:07.164418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5b75-a629-4125-887a-660bae726ac5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:07.170384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:07.171461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.172088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.172587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.174254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:07.175872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:07.203859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 968/1941] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:07.207705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:07.224080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08013c03-1ea6-40a8-978e-c857579682bb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:07.240809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:07.244190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.244619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.244906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] PATH_INFO: `/v3/projects/8cae06786c4845bab7927200f9dc48b2/users/753b62f8f2d24e66b5dba752b8797c3d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.277801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cae06786c4845bab7927200f9dc48b2, user_id=753b62f8f2d24e66b5dba752b8797c3d, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:07.280508 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:07.312286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 974/1942] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:07 2022] PUT /identity/v3/projects/8cae06786c4845bab7927200f9dc48b2/users/753b62f8f2d24e66b5dba752b8797c3d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:07.316916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:07.328630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89483c-45ba-46b5-94a5-db24a86001ef admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:07.368467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:07.400344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:07.421818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:07.423969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.424550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.425286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.427198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:07.429276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:07.433354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:07.443636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 969/1943] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:07.459670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:07.473591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b002c7-1713-4fb1-886c-453c6425b1ea admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:07.473714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7e3c7-2aae-4fbd-8cf2-bc4eea8a47c5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:07.479640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:07.480391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.480618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.480868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] PATH_INFO: `/v3/users/1d0c8a9a96a048529b9b0cb4a68bc348` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.482122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0c8a9a96a048529b9b0cb4a68bc348)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:07.483633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967a922-d4a0-484f-9d44-7749161057cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:07.484564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967a922-d4a0-484f-9d44-7749161057cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.484936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967a922-d4a0-484f-9d44-7749161057cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.485267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967a922-d4a0-484f-9d44-7749161057cc admin admin] PATH_INFO: `/v3/projects/8cae06786c4845bab7927200f9dc48b2/users/753b62f8f2d24e66b5dba752b8797c3d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.488153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:07.488296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967a922-d4a0-484f-9d44-7749161057cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cae06786c4845bab7927200f9dc48b2, user_id=753b62f8f2d24e66b5dba752b8797c3d, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:07.492061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967a922-d4a0-484f-9d44-7749161057cc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:07.510631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 970/1944] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:07 2022] PUT /identity/v3/projects/8cae06786c4845bab7927200f9dc48b2/users/753b62f8f2d24e66b5dba752b8797c3d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:07.571325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0c8a9a96a048529b9b0cb4a68bc348'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:07.578862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0c8a9a96a048529b9b0cb4a68bc348'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:07.591359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0c8a9a96a048529b9b0cb4a68bc348'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:07.592207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0c8a9a96a048529b9b0cb4a68bc348'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:07.605532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0c8a9a96a048529b9b0cb4a68bc348'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:07.612771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 975/1945] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:07 2022] DELETE /identity/v3/users/1d0c8a9a96a048529b9b0cb4a68bc348 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:57:07.617353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.617711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.618054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.641968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:07.643197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:07.717020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 971/1946] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:07.727117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:07.728628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:07.794256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357985f-cc22-4d51-a773-f92841f4e8a1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:07.801035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:07.802078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:07.802567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:07.803060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:07.819806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:07.822572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:07.837534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 976/1947] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:08.455105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:08.457117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:08.472707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b176c-21a3-4ddc-a901-674444d190ec None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:08.479537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:08.480369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:08.480971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:08.481342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] PATH_INFO: `/v3/projects/2d07fd52c5184080b044da1f1ced2f2a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:08.482874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d07fd52c5184080b044da1f1ced2f2a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:08.485342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:08.560776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d07fd52c5184080b044da1f1ced2f2a', 'request_id': 'req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.572989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d07fd52c5184080b044da1f1ced2f2a', 'request_id': 'req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.582065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d07fd52c5184080b044da1f1ced2f2a', 'request_id': 'req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.597346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] The token cache is being invalidate because project 2d07fd52c5184080b044da1f1ced2f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:08.597623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.598368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.600026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 972/1948] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:08 2022] DELETE /identity/v3/projects/2d07fd52c5184080b044da1f1ced2f2a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:57:08.604906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:08.617388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75863d24-a130-4ef4-bd92-d2ac6a1524d3 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:08.709460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:08.710172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:08.710516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:08.710856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] PATH_INFO: `/v3/users/fdfadf6312cb4fbbad5555689323fe81` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:08.712234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdfadf6312cb4fbbad5555689323fe81)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:08.714558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:08.720869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:08.786397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdfadf6312cb4fbbad5555689323fe81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.796167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdfadf6312cb4fbbad5555689323fe81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.809779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdfadf6312cb4fbbad5555689323fe81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.810911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdfadf6312cb4fbbad5555689323fe81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.831503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-71667a54-ecd9-40d2-9b34-374a6c215ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdfadf6312cb4fbbad5555689323fe81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:08.844346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 977/1949] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:08 2022] DELETE /identity/v3/users/fdfadf6312cb4fbbad5555689323fe81 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:09.508539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:09.510005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:09.524329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8e83c-02e9-403b-a3d9-cf58079cf9bf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:09.604709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:09.616700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:09.617431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:09.618032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:09.618838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] PATH_INFO: `/v3/projects/7184db17d1714ace8151b5a7d917a4f0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:09.621846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7184db17d1714ace8151b5a7d917a4f0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:09.625260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:09.749947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7184db17d1714ace8151b5a7d917a4f0', 'request_id': 'req-e826bacb-2d2f-44b4-9dda-de5ac71e3104'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:09.764689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7184db17d1714ace8151b5a7d917a4f0', 'request_id': 'req-e826bacb-2d2f-44b4-9dda-de5ac71e3104'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:09.773025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7184db17d1714ace8151b5a7d917a4f0', 'request_id': 'req-e826bacb-2d2f-44b4-9dda-de5ac71e3104'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:09.792303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] The token cache is being invalidate because project 7184db17d1714ace8151b5a7d917a4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:09.792617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:09.793356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e826bacb-2d2f-44b4-9dda-de5ac71e3104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:09.795978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 973/1950] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:09 2022] DELETE /identity/v3/projects/7184db17d1714ace8151b5a7d917a4f0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:09.801341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43708654-819e-4a87-9424-1b268263d1be None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:09.801813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43708654-819e-4a87-9424-1b268263d1be None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:09.801813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43708654-819e-4a87-9424-1b268263d1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:09.830303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-43708654-819e-4a87-9424-1b268263d1be None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:09.831540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-43708654-819e-4a87-9424-1b268263d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:09.990926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 978/1951] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:09.991903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be76a0-c703-4a80-884a-d49192f5ee32 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:09.992194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be76a0-c703-4a80-884a-d49192f5ee32 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:09.992498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be76a0-c703-4a80-884a-d49192f5ee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.022348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e1be76a0-c703-4a80-884a-d49192f5ee32 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:10.023611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e1be76a0-c703-4a80-884a-d49192f5ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:10.060696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 974/1952] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:10.067686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43708654-819e-4a87-9424-1b268263d1be None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:10.090878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43708654-819e-4a87-9424-1b268263d1be None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:10.100495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:10.101126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:10.101405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:10.101677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.102335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:10.102637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:10.105914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:10.136045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 979/1953] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:10.141331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be76a0-c703-4a80-884a-d49192f5ee32 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:10.156192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be76a0-c703-4a80-884a-d49192f5ee32 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:10.166137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:10.167919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:10.168454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:10.168564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.172433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:10.175949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:10.222705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 975/1954] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:10 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:10.227317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:10.269013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40641a06-aae9-432e-bd14-2b3a1082e158 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:10.403972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:10.404639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:10.404924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:10.405201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.406432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:10.414035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:10.465744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:10.472433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 980/1955] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:10 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:10.474743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:10.507248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee4a6-c9e2-476c-95d6-bec62460f671 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:10.528621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:10.529621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:10.532007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:10.532412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.533284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:10.539591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:10.583255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 976/1956] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:10.584922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:10.599633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f569e-1831-438b-beb9-ffcd0ebfd5cc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:10.626121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987926ee-5deb-4e6b-875b-d5b5f3a6cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:10.629780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987926ee-5deb-4e6b-875b-d5b5f3a6cf46 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:10.630099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987926ee-5deb-4e6b-875b-d5b5f3a6cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:10.630386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987926ee-5deb-4e6b-875b-d5b5f3a6cf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.631086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987926ee-5deb-4e6b-875b-d5b5f3a6cf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:10.632828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987926ee-5deb-4e6b-875b-d5b5f3a6cf46 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:10.661257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:10.662492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 981/1957] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:10.672482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e26fa-2a86-4e3d-8f8f-274ec2e95233 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:10.691191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:10.691949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:10.692256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:10.692879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] PATH_INFO: `/v3/projects/1403e88e53904798af55a44d20ecc546/users/ab507625d0d4448cb47f83fe4392e44f/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.729182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1403e88e53904798af55a44d20ecc546, user_id=ab507625d0d4448cb47f83fe4392e44f, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:10.731442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:10.750079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 977/1958] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:10 2022] PUT /identity/v3/projects/1403e88e53904798af55a44d20ecc546/users/ab507625d0d4448cb47f83fe4392e44f/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:10.761892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870372-f246-4c57-831d-9264d847f396 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:10.762242 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870372-f246-4c57-831d-9264d847f396 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:10.763043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8870372-f246-4c57-831d-9264d847f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:10.829898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d8870372-f246-4c57-831d-9264d847f396 None None] MFA Rules not processed for user `ab507625d0d4448cb47f83fe4392e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:10.831828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d8870372-f246-4c57-831d-9264d847f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:10.922772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:10.925363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 982/1959] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:11.084019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53174e2-e6f2-4c35-80bd-190cf39f3bdf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:11.090040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:11.090957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:11.091592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:11.092078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:11.093039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:11.096546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:11.176402 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 978/1960] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:10 2022] POST /identity/v3/projects => generated 381 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:11.183146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8870372-f246-4c57-831d-9264d847f396 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:11.211190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8870372-f246-4c57-831d-9264d847f396 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:11.374368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:11.375072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:11.375352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:11.377919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:11.378701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:11.383843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:11.431832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 983/1961] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:11 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:11.435739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:11.449722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a4730-2248-414e-aa71-5ab8f14ec76d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:11.461716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:11.462362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:11.462645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:11.462922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:11.464326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:11.467139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:11.474666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:11.498315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 979/1962] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:11.499820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:11.516110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5a40-90b9-4155-9769-7f572e6ce9b4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:11.529489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:11.529963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:11.530245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:11.530626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] PATH_INFO: `/v3/projects/6b06fa986a0f487792de698d326a8a7c/users/7ae3d17a36cd48c69c1262cdcdad12da/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:11.570504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b06fa986a0f487792de698d326a8a7c, user_id=7ae3d17a36cd48c69c1262cdcdad12da, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:11.576236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:11.605502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 984/1963] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:11 2022] PUT /identity/v3/projects/6b06fa986a0f487792de698d326a8a7c/users/7ae3d17a36cd48c69c1262cdcdad12da/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:11.610856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:11.630885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c38b3-1808-44b2-821c-6a261e2d543d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:11.800729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2256c01-2c35-4306-b857-4bc58409293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:11.801823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2256c01-2c35-4306-b857-4bc58409293e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:11.802311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2256c01-2c35-4306-b857-4bc58409293e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:11.802734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2256c01-2c35-4306-b857-4bc58409293e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:11.803540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2256c01-2c35-4306-b857-4bc58409293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:11.819337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2256c01-2c35-4306-b857-4bc58409293e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:11.842088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:11.855680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 980/1964] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:57:11.869487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea286deb-65a1-4d72-896a-042b434d2058 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:11.885457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:11.887848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:11.888816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:11.889160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] PATH_INFO: `/v3/projects/6b06fa986a0f487792de698d326a8a7c/users/7ae3d17a36cd48c69c1262cdcdad12da/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:11.892449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b06fa986a0f487792de698d326a8a7c, user_id=7ae3d17a36cd48c69c1262cdcdad12da, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:11.898568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:11.919290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 985/1965] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:11 2022] PUT /identity/v3/projects/6b06fa986a0f487792de698d326a8a7c/users/7ae3d17a36cd48c69c1262cdcdad12da/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:11.933422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854754d6-7275-46e1-9907-500eaf3b1f83 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:11.933818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854754d6-7275-46e1-9907-500eaf3b1f83 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:11.934168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854754d6-7275-46e1-9907-500eaf3b1f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:11.975855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-854754d6-7275-46e1-9907-500eaf3b1f83 None None] MFA Rules not processed for user `7ae3d17a36cd48c69c1262cdcdad12da`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:11.981265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-854754d6-7275-46e1-9907-500eaf3b1f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:12.078510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 981/1966] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:12.091115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:12.093538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:12.154435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:12.223054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29122b-8b26-4678-9004-d4290deeb6af admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:12.238514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:12.239305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:12.239589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:12.239870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:12.271281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:12.272892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:12.295106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 986/1967] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:13.043255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854754d6-7275-46e1-9907-500eaf3b1f83 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:13.062292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854754d6-7275-46e1-9907-500eaf3b1f83 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:13.076809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ae3d17a36cd48c69c1262cdcdad12da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b06fa986a0f487792de698d326a8a7c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:13.077710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:13.078203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:13.078650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:13.079825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:13.080721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:13.084535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:13.113598 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 982/1968] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:13.117936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:13.131854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6732a-3be3-4e5d-9fb9-e1f52417e26a service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:13.137606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ae3d17a36cd48c69c1262cdcdad12da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b06fa986a0f487792de698d326a8a7c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:13.138658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:13.139082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:13.140025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:13.140970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:13.152404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:13.206487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 987/1969] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:13 2022] POST /identity/v3/projects => generated 355 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:13.207378 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:13.226236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee01325-a013-4f53-b3ef-8d6cfb6f933f tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:13.457697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ae3d17a36cd48c69c1262cdcdad12da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b06fa986a0f487792de698d326a8a7c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:13.458468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:13.458885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:13.459230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:13.460858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:13.461247 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:13.463243 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:13.473126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:13.489275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 983/1970] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:57:13.548182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:13.566646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9b985-4859-4b5e-b44d-74c1ea9eaf1e tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:13.575942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ae3d17a36cd48c69c1262cdcdad12da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b06fa986a0f487792de698d326a8a7c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:13.576956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:13.577410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:13.577837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] PATH_INFO: `/v3/projects/66cc922cb0564f5a843eab12de2c1d69` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:13.579739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorizing `identity:delete_project(project_id=66cc922cb0564f5a843eab12de2c1d69)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:13.581925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:13.650526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cc922cb0564f5a843eab12de2c1d69', 'request_id': 'req-337ee9a2-4059-4a5e-8f89-c135ea1ff592'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:13.658160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cc922cb0564f5a843eab12de2c1d69', 'request_id': 'req-337ee9a2-4059-4a5e-8f89-c135ea1ff592'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:13.665817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cc922cb0564f5a843eab12de2c1d69', 'request_id': 'req-337ee9a2-4059-4a5e-8f89-c135ea1ff592'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:13.683336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] The token cache is being invalidate because project 66cc922cb0564f5a843eab12de2c1d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:13.683886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:13.684681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-337ee9a2-4059-4a5e-8f89-c135ea1ff592 tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:13.686532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 988/1971] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:13 2022] DELETE /identity/v3/projects/66cc922cb0564f5a843eab12de2c1d69 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:14.088469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:14.090446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:14.240744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b608ce7-590f-4abb-a750-f21b24813b5a tempest-VolumeQuotaClassesTest-1233593196 tempest-VolumeQuotaClassesTest-1233593196-project-admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:14.367082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:14.367753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:14.368030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:14.368324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] PATH_INFO: `/v3/users/ab507625d0d4448cb47f83fe4392e44f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:14.372081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab507625d0d4448cb47f83fe4392e44f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:14.378016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:14.459437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab507625d0d4448cb47f83fe4392e44f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:14.478315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab507625d0d4448cb47f83fe4392e44f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:14.488352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab507625d0d4448cb47f83fe4392e44f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:14.489517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab507625d0d4448cb47f83fe4392e44f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:14.490133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab507625d0d4448cb47f83fe4392e44f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:14.507517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 984/1972] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:14 2022] DELETE /identity/v3/users/ab507625d0d4448cb47f83fe4392e44f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:14.511996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a60d2-1840-4755-9c6f-47921723cdfe None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:14.512464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a60d2-1840-4755-9c6f-47921723cdfe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:14.512562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a60d2-1840-4755-9c6f-47921723cdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:14.556279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c32a60d2-1840-4755-9c6f-47921723cdfe None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:14.557699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c32a60d2-1840-4755-9c6f-47921723cdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:14.635724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 989/1973] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:14.647727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:14.649185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:14.682006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:14.718668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b823c-a8bb-453a-b7fe-9211dd41ea88 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:14.722545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374540ea-54b3-4bf8-9599-10cdb0d67a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:14.723157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374540ea-54b3-4bf8-9599-10cdb0d67a31 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:14.723437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374540ea-54b3-4bf8-9599-10cdb0d67a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:14.723715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374540ea-54b3-4bf8-9599-10cdb0d67a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:14.733499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374540ea-54b3-4bf8-9599-10cdb0d67a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:14.734938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374540ea-54b3-4bf8-9599-10cdb0d67a31 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:14.745267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 985/1974] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:14.893258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:14.893603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:14.893944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:14.894664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff851f-ed4f-42cb-8808-b740744b5add None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:14.895152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff851f-ed4f-42cb-8808-b740744b5add None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:14.895537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff851f-ed4f-42cb-8808-b740744b5add None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:14.897463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 986/1975] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 01:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:57:14.912305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:14.912720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:14.913094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:14.961848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:14.967021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:14.972618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] MFA Rules not processed for user `753b62f8f2d24e66b5dba752b8797c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:14.973886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.028612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:15.105702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 990/1976] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:15.106647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:15.107913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.119122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 987/1977] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 01:57:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:15.134398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:15.140166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5243e-efe8-4394-8ae7-61766bf0fa15 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:15.140507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.152299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:15.159680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:15.160415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:15.160872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] PATH_INFO: `/v3/users/cc7381cfffee4842bee1950a18caf807` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:15.163011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7381cfffee4842bee1950a18caf807)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:15.168468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:15.230798 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7381cfffee4842bee1950a18caf807'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.241373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7381cfffee4842bee1950a18caf807'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.243855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a3e2b-9d8d-41c3-a8e8-908552ee506a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:15.250650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:15.250866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7381cfffee4842bee1950a18caf807'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.252462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7381cfffee4842bee1950a18caf807'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.253007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:15.253561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:15.254121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:15.256755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.272907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7381cfffee4842bee1950a18caf807'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.282833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 991/1978] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:15 2022] DELETE /identity/v3/users/cc7381cfffee4842bee1950a18caf807 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:15.283704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:15.285645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.366976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:15.369246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:15.375842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:15.384356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 988/1979] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:15.385825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:15.387489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.403994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8a42e-35be-47fb-992c-3d6375512f98 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:15.445557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7146c1d-bbe4-4ed3-8506-4c0055ad46ed admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:15.470484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:15.471128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:15.471411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:15.471809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:15.481022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.535596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:15.537363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:15.537794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:15.538666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:15.538967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:15.539316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] PATH_INFO: `/v3/projects/1403e88e53904798af55a44d20ecc546` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:15.543433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1403e88e53904798af55a44d20ecc546)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:15.545622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:15.569926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 992/1980] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:15.603317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:15.605334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.639119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927c565-9e79-4db8-9697-482c20d973e0 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:15.645715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1403e88e53904798af55a44d20ecc546', 'request_id': 'req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.673440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1403e88e53904798af55a44d20ecc546', 'request_id': 'req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.689952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1403e88e53904798af55a44d20ecc546', 'request_id': 'req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.693483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:15.705925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] The token cache is being invalidate because project 1403e88e53904798af55a44d20ecc546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:15.706312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.707744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.713683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 989/1981] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:15 2022] DELETE /identity/v3/projects/1403e88e53904798af55a44d20ecc546 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:15.732646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:15.735215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.755046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:15.756319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:15.759068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:15.760331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] PATH_INFO: `/v3/projects/f4a7212d15b6463f834e8d1cc4ebf685` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:15.762713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a7212d15b6463f834e8d1cc4ebf685)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:15.766169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:15.874533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a7212d15b6463f834e8d1cc4ebf685', 'request_id': 'req-4e11e9e7-12fb-48fe-8699-758d2182c7d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.888334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a7212d15b6463f834e8d1cc4ebf685', 'request_id': 'req-4e11e9e7-12fb-48fe-8699-758d2182c7d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.899142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce75394-66b2-43ec-9c79-b5b9cc7f07c7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:15.903977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a7212d15b6463f834e8d1cc4ebf685', 'request_id': 'req-4e11e9e7-12fb-48fe-8699-758d2182c7d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.921381 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] The token cache is being invalidate because project f4a7212d15b6463f834e8d1cc4ebf685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:15.922550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.924859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e11e9e7-12fb-48fe-8699-758d2182c7d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:15.931549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 993/1982] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:15 2022] DELETE /identity/v3/projects/f4a7212d15b6463f834e8d1cc4ebf685 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:15.933776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bb11e-bf64-4cb9-8540-ea5aa5192480 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:15.934065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bb11e-bf64-4cb9-8540-ea5aa5192480 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:15.934358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bb11e-bf64-4cb9-8540-ea5aa5192480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:15.958459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-188bb11e-bf64-4cb9-8540-ea5aa5192480 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:15.959758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-188bb11e-bf64-4cb9-8540-ea5aa5192480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:15.980226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:16.020040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.020766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.021052 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.021326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] PATH_INFO: `/v3/users/7ae3d17a36cd48c69c1262cdcdad12da` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.022499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae3d17a36cd48c69c1262cdcdad12da)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.024519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.127623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae3d17a36cd48c69c1262cdcdad12da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.135269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 994/1983] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:16.136879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae58b3-845f-4c79-a3b1-8feaf0169607 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.137893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae58b3-845f-4c79-a3b1-8feaf0169607 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.138185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae58b3-845f-4c79-a3b1-8feaf0169607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.150876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae3d17a36cd48c69c1262cdcdad12da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.159681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae3d17a36cd48c69c1262cdcdad12da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.160950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae3d17a36cd48c69c1262cdcdad12da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.161901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae3d17a36cd48c69c1262cdcdad12da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.166782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7bae58b3-845f-4c79-a3b1-8feaf0169607 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:16.168363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7bae58b3-845f-4c79-a3b1-8feaf0169607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:16.181164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 990/1984] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:15 2022] DELETE /identity/v3/users/7ae3d17a36cd48c69c1262cdcdad12da => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:16.191270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 995/1985] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:16.195741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.276502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a81473-b15f-42c5-85d0-c93fb4906087 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:16.285816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.286147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.286527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.286904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.287724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:16.288134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.294633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.312328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 991/1986] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:16.313285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae58b3-845f-4c79-a3b1-8feaf0169607 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.326749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae58b3-845f-4c79-a3b1-8feaf0169607 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:16.334250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.334898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.335178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.335455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.336164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.338195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.378436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 996/1987] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:16 2022] POST /identity/v3/projects => generated 391 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:16.379565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.393248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aab3f-a24b-4906-bcd6-5d56d454e286 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:16.459166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:16.473970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.475011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.475545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.476107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.477509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.480284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.518051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 992/1988] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:16 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:16.521855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.537276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fdfd5-73aa-4804-b57f-664dcff7e4d5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:16.542990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.543932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.544347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.544735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.545703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.548170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.562730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 997/1989] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:57:16.564495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.572178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.574392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:16.587940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b854f7-7afd-42ae-a7e3-877cf6af8982 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:16.594203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.595830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.596454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.605795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.605918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192eb6-0a93-4050-be21-9fcd7a2ddb9e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:16.606865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.609707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.611770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.612724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.613098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.613487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] PATH_INFO: `/v3/projects/6b06fa986a0f487792de698d326a8a7c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.614874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b06fa986a0f487792de698d326a8a7c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.616968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.625743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 993/1990] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:16.630033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:16.631032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.642946 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b5da3-1af3-452a-8d7f-062c9bde7ee4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:16.651930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ebf06-ae4a-4d16-9a9b-27ea075f23f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:16.652828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ebf06-ae4a-4d16-9a9b-27ea075f23f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.653172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ebf06-ae4a-4d16-9a9b-27ea075f23f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.654150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ebf06-ae4a-4d16-9a9b-27ea075f23f5 admin admin] PATH_INFO: `/v3/projects/688f10cbc5ce42b895bb7a9285749cca/users/86f364f03c7d4360b4a75ed194846ade/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.682661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ebf06-ae4a-4d16-9a9b-27ea075f23f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688f10cbc5ce42b895bb7a9285749cca, user_id=86f364f03c7d4360b4a75ed194846ade, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:16.685656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ebf06-ae4a-4d16-9a9b-27ea075f23f5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:16.698010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b06fa986a0f487792de698d326a8a7c', 'request_id': 'req-80224901-963d-41cb-b207-169e6b0f588f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.709710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 994/1991] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:16 2022] PUT /identity/v3/projects/688f10cbc5ce42b895bb7a9285749cca/users/86f364f03c7d4360b4a75ed194846ade/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:16.713482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b06fa986a0f487792de698d326a8a7c', 'request_id': 'req-80224901-963d-41cb-b207-169e6b0f588f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.714531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.714827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.715123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.722286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b06fa986a0f487792de698d326a8a7c', 'request_id': 'req-80224901-963d-41cb-b207-169e6b0f588f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.749797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:16.751657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:16.772637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] The token cache is being invalidate because project 6b06fa986a0f487792de698d326a8a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:16.773124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.774037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-80224901-963d-41cb-b207-169e6b0f588f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:16.776355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 998/1992] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:16 2022] DELETE /identity/v3/projects/6b06fa986a0f487792de698d326a8a7c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:16.781921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4886-6892-41ce-873b-b08c7d8bc4d1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.782244 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4886-6892-41ce-873b-b08c7d8bc4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.783078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4886-6892-41ce-873b-b08c7d8bc4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.827226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-971e4886-6892-41ce-873b-b08c7d8bc4d1 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:16.830715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-971e4886-6892-41ce-873b-b08c7d8bc4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:16.871664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 995/1993] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:16.872727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:16.875297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:16.962125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 999/1994] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:16.970024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bfc3a-3f0f-4511-96d5-f80b1f079c04 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:16.970310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bfc3a-3f0f-4511-96d5-f80b1f079c04 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:16.970746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bfc3a-3f0f-4511-96d5-f80b1f079c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:16.999656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-253bfc3a-3f0f-4511-96d5-f80b1f079c04 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:17.000892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-253bfc3a-3f0f-4511-96d5-f80b1f079c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:17.004603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fb80f-1240-4b92-8fec-7479cb2c03f7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.008987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.009945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.010453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.010948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.014671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1000/1995] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:17.016540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bfc3a-3f0f-4511-96d5-f80b1f079c04 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.028097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bfc3a-3f0f-4511-96d5-f80b1f079c04 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.033803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.035007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.035007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.035268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.035883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:17.036181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.036933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.037816 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.040253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.051659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1001/1996] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:17.055698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.066223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 996/1997] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:17.080170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79e95b-14b7-48cf-bea7-66e80adb6e39 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.090315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.091066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.091220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.091496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.092141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.094417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.177049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.178075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1002/1998] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:17 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:17.193593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b8ab-3e95-486d-b6d3-c052a806a315 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.288472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:17.314931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.317116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.317641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.318222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.319259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.326249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.374975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 997/1999] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:17 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:17.377317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.391116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38992742-81a1-4cad-8459-d91ee4603a1c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.399749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.400690 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.401072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.401533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.402296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.412641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.432939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1003/2000] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:57:17.436814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.451183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ce057-488f-491c-b04d-b13a40c9019c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.465623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68a718-7d0a-4cb4-86d1-702afac5e5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.465623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68a718-7d0a-4cb4-86d1-702afac5e5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.465878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68a718-7d0a-4cb4-86d1-702afac5e5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.465953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68a718-7d0a-4cb4-86d1-702afac5e5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.466725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab68a718-7d0a-4cb4-86d1-702afac5e5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.468825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab68a718-7d0a-4cb4-86d1-702afac5e5c1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.492267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 998/2001] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:17.493538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.506899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92000d66-f54c-4f4e-8315-cd8920e9f7e8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.513433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416bf29-814d-4811-8322-b1193d58798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.514901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416bf29-814d-4811-8322-b1193d58798e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.515216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416bf29-814d-4811-8322-b1193d58798e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.515498 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416bf29-814d-4811-8322-b1193d58798e admin admin] PATH_INFO: `/v3/projects/4b7f511a64b248acbc92de43e21c30ee/users/d594fdf5c68040dabba76be54f8850eb/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.550443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416bf29-814d-4811-8322-b1193d58798e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7f511a64b248acbc92de43e21c30ee, user_id=d594fdf5c68040dabba76be54f8850eb, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.555709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416bf29-814d-4811-8322-b1193d58798e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.574355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa4c2-ea04-475b-ae16-0be9501042d8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.574540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa4c2-ea04-475b-ae16-0be9501042d8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.574818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aa4c2-ea04-475b-ae16-0be9501042d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.593926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1004/2002] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:17 2022] PUT /identity/v3/projects/4b7f511a64b248acbc92de43e21c30ee/users/d594fdf5c68040dabba76be54f8850eb/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:17.596003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca371a-8cbd-40da-b3a0-95f3be639edf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.596330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca371a-8cbd-40da-b3a0-95f3be639edf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.596610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca371a-8cbd-40da-b3a0-95f3be639edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.601563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-283aa4c2-ea04-475b-ae16-0be9501042d8 None None] MFA Rules not processed for user `1fafe6ce050545b8870f88d4ae0aa1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:17.616644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-283aa4c2-ea04-475b-ae16-0be9501042d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:17.648734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f9ca371a-8cbd-40da-b3a0-95f3be639edf None None] MFA Rules not processed for user `d594fdf5c68040dabba76be54f8850eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:17.650143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f9ca371a-8cbd-40da-b3a0-95f3be639edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:17.672641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:17.694508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 999/2003] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:17.777634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1005/2004] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:17.781983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aa4c2-ea04-475b-ae16-0be9501042d8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.917317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aa4c2-ea04-475b-ae16-0be9501042d8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:17.922135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:17.922765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:17.923103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:17.923370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:17.924013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:17.926137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:17.973468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1000/2005] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:17 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:17.974916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca371a-8cbd-40da-b3a0-95f3be639edf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:17.990889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca371a-8cbd-40da-b3a0-95f3be639edf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:18.089465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:18.090096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.090368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.090640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.091448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:18.093625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:18.133914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1006/2006] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:17 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:18.138618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:18.151274 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69b0d-20a6-4746-9f03-8014efb58a42 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:18.157080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:18.158253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.162754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.164080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.164923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:18.166737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:18.182187 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1001/2007] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:18.182973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:18.195845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527c544-93ae-4cb1-b821-a164b180801b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:18.204602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:18.205299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.205579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.205856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] PATH_INFO: `/v3/projects/ae839420add148a992993d93023cfcea/users/8df3f15243064cc298e900cb14bd3b17/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.230119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae839420add148a992993d93023cfcea, user_id=8df3f15243064cc298e900cb14bd3b17, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:18.234325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:18.260837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1007/2008] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:18 2022] PUT /identity/v3/projects/ae839420add148a992993d93023cfcea/users/8df3f15243064cc298e900cb14bd3b17/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:57:18.260984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:18.277285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c62679c-4179-4c12-a203-75095292d738 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:18.395240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:18.425396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def84d9-544a-4228-8138-bf1ebeae17c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:18.426035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def84d9-544a-4228-8138-bf1ebeae17c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.426321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def84d9-544a-4228-8138-bf1ebeae17c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.426599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def84d9-544a-4228-8138-bf1ebeae17c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.427260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def84d9-544a-4228-8138-bf1ebeae17c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:18.428933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def84d9-544a-4228-8138-bf1ebeae17c4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:18.446918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1002/2009] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:18.450913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:18.469433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c55904-5f29-4302-b0a6-043464f47792 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:18.476353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:18.476955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.477251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.477529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] PATH_INFO: `/v3/projects/ae839420add148a992993d93023cfcea/users/8df3f15243064cc298e900cb14bd3b17/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.480673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae839420add148a992993d93023cfcea, user_id=8df3f15243064cc298e900cb14bd3b17, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:18.483384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:18.504427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1008/2010] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:18 2022] PUT /identity/v3/projects/ae839420add148a992993d93023cfcea/users/8df3f15243064cc298e900cb14bd3b17/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:18.514580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.514877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.515160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.532840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:18.538539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:18.586418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] MFA Rules not processed for user `8df3f15243064cc298e900cb14bd3b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:18.587658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:18.681550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d243b-2433-4c64-a174-ab9ec88b0993 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:18.688089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:18.689940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.690495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.690885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] PATH_INFO: `/v3/users/47e66354f7614713b98d8e6a84a35823` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.692304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e66354f7614713b98d8e6a84a35823)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:18.694328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:18.697500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:18.725866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1003/2011] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:18.737153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:18.739943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:18.798954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e66354f7614713b98d8e6a84a35823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:18.823919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e66354f7614713b98d8e6a84a35823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:18.851011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e66354f7614713b98d8e6a84a35823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:18.857289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e66354f7614713b98d8e6a84a35823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:18.859376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090681cf-7327-4111-b316-9eb96066acc1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:18.865819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:18.866214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.866568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.866852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.868976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:18.896974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2d4721a2-5413-476b-a808-211d28ce054d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e66354f7614713b98d8e6a84a35823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:18.922528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1009/2012] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:18 2022] DELETE /identity/v3/users/47e66354f7614713b98d8e6a84a35823 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:18.928926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:18.929377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:18.929544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:18.962428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:18.963651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:18.998062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:18.999627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:19.027430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1004/2013] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:57:19.074006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1010/2014] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:19.085761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:19.087761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:19.128777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:19.198920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632ff63-852c-46e7-9b95-2a3c921fa648 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:19.321005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85ad61-cac6-4eb1-a9ee-077d471aae73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:19.321696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ad61-cac6-4eb1-a9ee-077d471aae73 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:19.321859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ad61-cac6-4eb1-a9ee-077d471aae73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:19.322147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ad61-cac6-4eb1-a9ee-077d471aae73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:19.345321 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85ad61-cac6-4eb1-a9ee-077d471aae73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:19.347022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85ad61-cac6-4eb1-a9ee-077d471aae73 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:19.369702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1005/2015] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:57:19.920409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:19.921958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:19.935810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c62e6-b10a-430f-ba6d-ec6ed132c723 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:19.949949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:19.950634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:19.950915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:19.951188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] PATH_INFO: `/v3/projects/27de5ef7e8844ea88d6cf717b0da1ebc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:19.952431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27de5ef7e8844ea88d6cf717b0da1ebc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:19.954929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.009345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27de5ef7e8844ea88d6cf717b0da1ebc', 'request_id': 'req-f5979f20-299f-4692-8187-de0cb010312c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:20.015560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27de5ef7e8844ea88d6cf717b0da1ebc', 'request_id': 'req-f5979f20-299f-4692-8187-de0cb010312c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:20.022764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27de5ef7e8844ea88d6cf717b0da1ebc', 'request_id': 'req-f5979f20-299f-4692-8187-de0cb010312c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:20.037176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] The token cache is being invalidate because project 27de5ef7e8844ea88d6cf717b0da1ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:20.037696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:20.038502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5979f20-299f-4692-8187-de0cb010312c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:20.040232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1011/2016] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:19 2022] DELETE /identity/v3/projects/27de5ef7e8844ea88d6cf717b0da1ebc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:57:20.045469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d2d4-db10-4615-816d-56786fdfc75a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.045755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d2d4-db10-4615-816d-56786fdfc75a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.046084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d2d4-db10-4615-816d-56786fdfc75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.070827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5627d2d4-db10-4615-816d-56786fdfc75a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:20.072169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5627d2d4-db10-4615-816d-56786fdfc75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:20.140791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1006/2017] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:20.149232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18f6af-0c48-4726-972b-b79cebafb094 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.149640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18f6af-0c48-4726-972b-b79cebafb094 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.149904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18f6af-0c48-4726-972b-b79cebafb094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.160120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:20.174441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dc18f6af-0c48-4726-972b-b79cebafb094 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:20.176087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dc18f6af-0c48-4726-972b-b79cebafb094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:20.184942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1012/2018] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:20.189783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d2d4-db10-4615-816d-56786fdfc75a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:20.199913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d2d4-db10-4615-816d-56786fdfc75a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:20.205849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:20.206774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.207232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.207520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.208227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:20.208726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:20.210543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.228206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1007/2019] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:20.231402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18f6af-0c48-4726-972b-b79cebafb094 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:20.241848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18f6af-0c48-4726-972b-b79cebafb094 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:20.246882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:20.247686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.248044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.248372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.249038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:20.252678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.294820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1013/2020] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:20 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:20.298487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:20.314330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb2a59-6c4a-47be-9fc9-36d750877f52 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:20.386633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:20.387292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.387571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.387844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.388535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:20.390486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.408545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:20.419233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1008/2021] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:20 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:20.423730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:20.447198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50748a30-1c0f-4df7-9654-77e7c6df1429 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:20.452120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:20.452777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.453058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.453334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.453993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:20.456037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.484831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1014/2022] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:20.489099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:20.502325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579551b6-ca40-4e5f-af6d-81f194edd24b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:20.511900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f19c85-0ac1-440e-9b8e-279a52304a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:20.517381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f19c85-0ac1-440e-9b8e-279a52304a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.518035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f19c85-0ac1-440e-9b8e-279a52304a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.519162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f19c85-0ac1-440e-9b8e-279a52304a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.520039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f19c85-0ac1-440e-9b8e-279a52304a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:20.521814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f19c85-0ac1-440e-9b8e-279a52304a5e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.544549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1009/2023] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:57:20.549525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:20.569066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e21e-3c2a-424c-9de4-28ed18c9a740 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:20.579544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:20.579544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.579544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.580062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] PATH_INFO: `/v3/projects/ffefe2fb4a02413ca10f96adc40aa820/users/de1557d78e66466b94fb0c197724734a/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.604505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffefe2fb4a02413ca10f96adc40aa820, user_id=de1557d78e66466b94fb0c197724734a, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:20.607273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.629735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1015/2024] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:20 2022] PUT /identity/v3/projects/ffefe2fb4a02413ca10f96adc40aa820/users/de1557d78e66466b94fb0c197724734a/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:20.641389 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7907-3921-4244-9310-ca3c2a00943d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.641857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7907-3921-4244-9310-ca3c2a00943d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.642235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa7907-3921-4244-9310-ca3c2a00943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.689525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d3aa7907-3921-4244-9310-ca3c2a00943d None None] MFA Rules not processed for user `de1557d78e66466b94fb0c197724734a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:20.702758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d3aa7907-3921-4244-9310-ca3c2a00943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:20.814722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1010/2025] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:20.826817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:20.829796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:20.934095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f86dd6-d5ee-4bbc-8ae0-8af8480f22de admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:20.944738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07333ddf-9e5b-47ad-8bcd-737fd37e0121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:20.944738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07333ddf-9e5b-47ad-8bcd-737fd37e0121 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:20.944738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07333ddf-9e5b-47ad-8bcd-737fd37e0121 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:20.944738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07333ddf-9e5b-47ad-8bcd-737fd37e0121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:20.956473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07333ddf-9e5b-47ad-8bcd-737fd37e0121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:20.957935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07333ddf-9e5b-47ad-8bcd-737fd37e0121 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:20.973962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1016/2026] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:22.568549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:22.569047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:22.569215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:22.605678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] MFA Rules not processed for user `86f364f03c7d4360b4a75ed194846ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:22.607381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:22.685835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1011/2027] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:22.697025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627086e-8e0b-4c79-a70e-0251aa0b646a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:22.697513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627086e-8e0b-4c79-a70e-0251aa0b646a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:22.697707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627086e-8e0b-4c79-a70e-0251aa0b646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:22.722544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b627086e-8e0b-4c79-a70e-0251aa0b646a None None] MFA Rules not processed for user `86f364f03c7d4360b4a75ed194846ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:22.723757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b627086e-8e0b-4c79-a70e-0251aa0b646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:22.730790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1017/2028] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:22.748621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:22.751590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:22.826397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:22.831913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a51a8-c307-4ac7-a709-9288d412e7cb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:22.838010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:22.838630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:22.838909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:22.839181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:22.849215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:22.850845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:22.868430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1012/2029] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:23.568619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef016121-2e26-4b8c-9cf3-557283b9bce9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:23.569476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef016121-2e26-4b8c-9cf3-557283b9bce9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:23.569831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef016121-2e26-4b8c-9cf3-557283b9bce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:23.606246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ef016121-2e26-4b8c-9cf3-557283b9bce9 None None] MFA Rules not processed for user `86f364f03c7d4360b4a75ed194846ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:23.608475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ef016121-2e26-4b8c-9cf3-557283b9bce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:23.621140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1018/2030] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:24.186272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:24.187736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:24.259824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00705-d4e9-4cd3-8f9c-06bd337afdeb service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:24.268829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:24.269575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:24.269899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:24.270205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] PATH_INFO: `/v3/users/d594fdf5c68040dabba76be54f8850eb` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:24.272000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d594fdf5c68040dabba76be54f8850eb)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:24.274683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:24.355666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd594fdf5c68040dabba76be54f8850eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:24.370019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd594fdf5c68040dabba76be54f8850eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:24.379142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd594fdf5c68040dabba76be54f8850eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:24.380272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd594fdf5c68040dabba76be54f8850eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:24.381069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd594fdf5c68040dabba76be54f8850eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:24.400324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1013/2031] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:24 2022] DELETE /identity/v3/users/d594fdf5c68040dabba76be54f8850eb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:24.415260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:24.415260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:24.415260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:24.427645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:24.432069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:24.499468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1019/2032] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:24.510084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:24.511559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:24.573096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac44aa4-60e7-4dbf-9f92-7b2cf6391836 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:24.576878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:24.577500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:24.577767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:24.578041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:24.588112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:24.589651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:24.599880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1014/2033] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:24.956299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:24.957789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:24.984320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fafa7b-2890-4b87-94da-62232e3fd633 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:24.997559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:24.998231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:24.998510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:24.998945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] PATH_INFO: `/v3/projects/4b7f511a64b248acbc92de43e21c30ee` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:25.000829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7f511a64b248acbc92de43e21c30ee)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:25.004754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:25.055262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:25.081284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7f511a64b248acbc92de43e21c30ee', 'request_id': 'req-6b86f388-221b-4c76-af10-2d4396b69d20'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.089298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7f511a64b248acbc92de43e21c30ee', 'request_id': 'req-6b86f388-221b-4c76-af10-2d4396b69d20'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.096448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7f511a64b248acbc92de43e21c30ee', 'request_id': 'req-6b86f388-221b-4c76-af10-2d4396b69d20'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.109492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] The token cache is being invalidate because project 4b7f511a64b248acbc92de43e21c30ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:25.109706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.111130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.112780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1020/2034] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:24 2022] DELETE /identity/v3/projects/4b7f511a64b248acbc92de43e21c30ee => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:25.117164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:25.128615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5e66-e50f-4dfc-b77e-c80059c8b181 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:25.147347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:25.224581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:25.225471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:25.227116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:25.227477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] PATH_INFO: `/v3/users/8df3f15243064cc298e900cb14bd3b17` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:25.228944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df3f15243064cc298e900cb14bd3b17)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:25.230914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:25.332339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df3f15243064cc298e900cb14bd3b17'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.350035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df3f15243064cc298e900cb14bd3b17'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.358280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df3f15243064cc298e900cb14bd3b17'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.359044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df3f15243064cc298e900cb14bd3b17'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.359810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f389d6ee-e4c7-44e9-a036-a9c9c6b0f630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df3f15243064cc298e900cb14bd3b17'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.374315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1015/2035] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:25 2022] DELETE /identity/v3/users/8df3f15243064cc298e900cb14bd3b17 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:25.703382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:25.705530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:25.726759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86f388-221b-4c76-af10-2d4396b69d20 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:25.790166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:25.790811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:25.791090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:25.791401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] PATH_INFO: `/v3/projects/ae839420add148a992993d93023cfcea` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:25.792527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae839420add148a992993d93023cfcea)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:25.794479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:25.846267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae839420add148a992993d93023cfcea', 'request_id': 'req-28e022d0-f33b-4dc5-8913-608339938fb6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.852787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae839420add148a992993d93023cfcea', 'request_id': 'req-28e022d0-f33b-4dc5-8913-608339938fb6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.858290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae839420add148a992993d93023cfcea', 'request_id': 'req-28e022d0-f33b-4dc5-8913-608339938fb6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.871489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] The token cache is being invalidate because project ae839420add148a992993d93023cfcea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:25.871785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.872478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-28e022d0-f33b-4dc5-8913-608339938fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:25.874053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1021/2036] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:25 2022] DELETE /identity/v3/projects/ae839420add148a992993d93023cfcea => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:25.879552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4162c31-98cb-4493-8116-0e37fe36dd20 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:25.879847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4162c31-98cb-4493-8116-0e37fe36dd20 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:25.880135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4162c31-98cb-4493-8116-0e37fe36dd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:25.902372 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f4162c31-98cb-4493-8116-0e37fe36dd20 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:25.903651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f4162c31-98cb-4493-8116-0e37fe36dd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:25.967908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1016/2037] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:25.975886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2a20-bd6d-41ef-9451-036d54e8d4ba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:25.976177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2a20-bd6d-41ef-9451-036d54e8d4ba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:25.977003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2a20-bd6d-41ef-9451-036d54e8d4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.004723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f1bc2a20-bd6d-41ef-9451-036d54e8d4ba None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:26.006041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f1bc2a20-bd6d-41ef-9451-036d54e8d4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:26.013992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1022/2038] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:26.018753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4162c31-98cb-4493-8116-0e37fe36dd20 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.028459 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4162c31-98cb-4493-8116-0e37fe36dd20 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.032937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.033539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.033813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.034084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.034725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:26.035023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.036635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.048096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1017/2039] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:26.052128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc2a20-bd6d-41ef-9451-036d54e8d4ba None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.061162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc2a20-bd6d-41ef-9451-036d54e8d4ba None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.066414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.067047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.067356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.067662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.068371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.070285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.098222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1023/2040] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:26 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:26.102163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.113516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d980c-d174-423b-a34c-dfc8c1d60525 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.132200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:26.185276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.185911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.186211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.186481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.187120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.189735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.211672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1018/2041] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:26 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:26.216015 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.226874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44d2c-1ca4-47ea-a2df-9a3881d5b346 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.231448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.232055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.232375 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.232652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.233300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.235002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.247902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1024/2042] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:26.249986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.261621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ef8b7-c70b-4460-bb8a-33f4d7af4bee admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.266405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e78a63-e16b-4656-8269-367cf8a1babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.267712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78a63-e16b-4656-8269-367cf8a1babd admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.267996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78a63-e16b-4656-8269-367cf8a1babd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.268285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e78a63-e16b-4656-8269-367cf8a1babd admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.268928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e78a63-e16b-4656-8269-367cf8a1babd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.270533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e78a63-e16b-4656-8269-367cf8a1babd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.282008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1019/2043] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:26.286432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.293668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:26.298405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f17bb-55f1-4056-8191-6e3c23ec0994 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.307051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.307738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.308017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.308319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] PATH_INFO: `/v3/projects/1fb0fc680f394e6c90d4558766ba577a/users/7e81d9321e554466b3f7ec833a6e0215/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.324507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb0fc680f394e6c90d4558766ba577a, user_id=7e81d9321e554466b3f7ec833a6e0215, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.327328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.341638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1025/2044] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:26 2022] PUT /identity/v3/projects/1fb0fc680f394e6c90d4558766ba577a/users/7e81d9321e554466b3f7ec833a6e0215/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:26.353033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e429-f532-40ec-8dd5-55893fcc37db None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.353322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e429-f532-40ec-8dd5-55893fcc37db None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.353601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e429-f532-40ec-8dd5-55893fcc37db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.383133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-82b3e429-f532-40ec-8dd5-55893fcc37db None None] MFA Rules not processed for user `7e81d9321e554466b3f7ec833a6e0215`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:26.384397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-82b3e429-f532-40ec-8dd5-55893fcc37db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:26.452326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1020/2045] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:26.457227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.522292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394200-3de1-4887-9ba7-b661fe7a5ae1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.527019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.527647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.527931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.528202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.528875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.530785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.562098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1026/2046] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:26 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:26.563215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e429-f532-40ec-8dd5-55893fcc37db None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.574950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e429-f532-40ec-8dd5-55893fcc37db None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.639442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.640156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.640592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.640951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.642147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.644051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.670777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1021/2047] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:26 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:26.672000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.682755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadef2-bd26-4b49-b85b-299466553aac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.687623 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.688892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.689237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.689572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.690318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.692073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.705109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1027/2048] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:26.709218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.719643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22674cc2-2148-49b1-a6ab-0f7d4ecbf7e3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.724592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d53bc7-bd53-4673-841e-e3c7e497070f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.725239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53bc7-bd53-4673-841e-e3c7e497070f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.725503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53bc7-bd53-4673-841e-e3c7e497070f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.725786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53bc7-bd53-4673-841e-e3c7e497070f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.726441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d53bc7-bd53-4673-841e-e3c7e497070f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.728072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d53bc7-bd53-4673-841e-e3c7e497070f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.739506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1022/2049] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:26.743745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.754756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2081d26-a25d-47cc-a720-e8e524d1820e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:26.759549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:26.760259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.760548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.760828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] PATH_INFO: `/v3/projects/092e63651c624980b16afa1067dd368d/users/c42e79e9e83e45778716f86e57efee0d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.777365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=092e63651c624980b16afa1067dd368d, user_id=c42e79e9e83e45778716f86e57efee0d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:26.780002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:26.793810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1028/2050] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:26 2022] PUT /identity/v3/projects/092e63651c624980b16afa1067dd368d/users/c42e79e9e83e45778716f86e57efee0d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:26.804850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a8e91-7c2a-4fc2-bed8-5d974b346f6c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:26.805266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a8e91-7c2a-4fc2-bed8-5d974b346f6c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:26.805556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a8e91-7c2a-4fc2-bed8-5d974b346f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:26.819938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:26.836170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a19a8e91-7c2a-4fc2-bed8-5d974b346f6c None None] MFA Rules not processed for user `c42e79e9e83e45778716f86e57efee0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:26.837417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a19a8e91-7c2a-4fc2-bed8-5d974b346f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:26.913433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1023/2051] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:26.918005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:26.999125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48de4c-8cd4-4ab5-8869-daff7a9e7668 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:27.005812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:27.006729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.007368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.008197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.009361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:27.011321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:27.050964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1029/2052] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:26 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:27.054836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a8e91-7c2a-4fc2-bed8-5d974b346f6c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:27.068473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a8e91-7c2a-4fc2-bed8-5d974b346f6c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:27.134555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:27.135175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.135482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.135761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.136451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:27.138365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:27.161019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1024/2053] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:27 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:27.165008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:27.177380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c285d-11d0-4426-8eb3-dcdb2e924c76 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:27.182913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:27.183645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.183991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.184376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.185096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:27.186795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:27.199686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1030/2054] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:27.204293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:27.215080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019775bb-dfe8-43e6-968d-4a5a96935378 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:27.220486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:27.221225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.221642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.221931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] PATH_INFO: `/v3/projects/2c2d468beb0d4aaf8287568fa65f3130/users/830bbd32106644cca5856d8ca731b84f/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.237712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2d468beb0d4aaf8287568fa65f3130, user_id=830bbd32106644cca5856d8ca731b84f, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:27.240526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:27.255567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1025/2055] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:27 2022] PUT /identity/v3/projects/2c2d468beb0d4aaf8287568fa65f3130/users/830bbd32106644cca5856d8ca731b84f/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:27.259480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:27.266121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:27.270674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992ad5-ae1a-433e-9f7a-04b105a1458d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:27.339431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896e9e1-2d9f-489d-8da3-b5152f7b5619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:27.340032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e9e1-2d9f-489d-8da3-b5152f7b5619 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.340454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e9e1-2d9f-489d-8da3-b5152f7b5619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.340727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e9e1-2d9f-489d-8da3-b5152f7b5619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.341361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896e9e1-2d9f-489d-8da3-b5152f7b5619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:27.342948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896e9e1-2d9f-489d-8da3-b5152f7b5619 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:27.356334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1031/2056] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:27.357548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:27.368062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd81abf-13c8-4dbb-86eb-dc1e0836377e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:27.372762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:27.373419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.373703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.373979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] PATH_INFO: `/v3/projects/2c2d468beb0d4aaf8287568fa65f3130/users/830bbd32106644cca5856d8ca731b84f/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.376634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2d468beb0d4aaf8287568fa65f3130, user_id=830bbd32106644cca5856d8ca731b84f, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:27.379523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:27.390517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1026/2057] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:27 2022] PUT /identity/v3/projects/2c2d468beb0d4aaf8287568fa65f3130/users/830bbd32106644cca5856d8ca731b84f/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:57:27.401646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4db05-3cf4-4c1f-a656-8fb10d7cef88 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.401936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4db05-3cf4-4c1f-a656-8fb10d7cef88 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.402217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4db05-3cf4-4c1f-a656-8fb10d7cef88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.431382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d0a4db05-3cf4-4c1f-a656-8fb10d7cef88 None None] MFA Rules not processed for user `830bbd32106644cca5856d8ca731b84f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:27.432635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d0a4db05-3cf4-4c1f-a656-8fb10d7cef88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:27.505268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1032/2058] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:27.516837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:27.518570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:27.574403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:27.578540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef162c5-0724-4542-9772-06142214bbcd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:27.583330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:27.583949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:27.584224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:27.584526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:27.645358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:27.646791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:27.656508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1027/2059] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:34.593713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4db05-3cf4-4c1f-a656-8fb10d7cef88 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:34.604877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4db05-3cf4-4c1f-a656-8fb10d7cef88 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:34.609475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:34.610256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:34.610573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:34.610844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:34.671314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:34.672843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:34.685024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1033/2060] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:35.378271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:35.380480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:35.392667 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9bddb-4d99-436f-aa5e-b39a7a210fff service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:35.397422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24cc1a-5eee-4487-9aba-829b4c230015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:35.397797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24cc1a-5eee-4487-9aba-829b4c230015 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:35.398434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24cc1a-5eee-4487-9aba-829b4c230015 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:35.398724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24cc1a-5eee-4487-9aba-829b4c230015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:35.408821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24cc1a-5eee-4487-9aba-829b4c230015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:35.410259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24cc1a-5eee-4487-9aba-829b4c230015 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:35.420141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1028/2061] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:36.238122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:36.239557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:36.268476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:36.304687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80ce19-a1b1-429a-8a18-5ce93dabd694 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:36.309200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:36.309817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:36.310093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:36.310364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] PATH_INFO: `/v3/users/de1557d78e66466b94fb0c197724734a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:36.311474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1557d78e66466b94fb0c197724734a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:36.313457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:36.359370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1557d78e66466b94fb0c197724734a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.366727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1557d78e66466b94fb0c197724734a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.374023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1557d78e66466b94fb0c197724734a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.374619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1557d78e66466b94fb0c197724734a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.386605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1557d78e66466b94fb0c197724734a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.392570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1034/2062] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:36 2022] DELETE /identity/v3/users/de1557d78e66466b94fb0c197724734a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:36.396984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:36.397837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:36.398123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:36.417390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:36.418874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:36.482966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1029/2063] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:36.493513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:36.494937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:36.553821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40103d00-3180-4362-b951-e9a0c4835320 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:36.557503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49af3f-a2fe-4de9-900c-b9a6e56b9ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:36.558105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49af3f-a2fe-4de9-900c-b9a6e56b9ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:36.558384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49af3f-a2fe-4de9-900c-b9a6e56b9ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:36.558851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49af3f-a2fe-4de9-900c-b9a6e56b9ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:36.569085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae49af3f-a2fe-4de9-900c-b9a6e56b9ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:36.570595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae49af3f-a2fe-4de9-900c-b9a6e56b9ab3 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:36.581906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1035/2064] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:36.880153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:36.881784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:36.898584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df681564-5f10-40e1-9e43-26392b1f19a4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:36.909807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:36.910857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:36.911490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:36.911770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] PATH_INFO: `/v3/projects/ffefe2fb4a02413ca10f96adc40aa820` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:36.913092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffefe2fb4a02413ca10f96adc40aa820)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:36.915203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:36.961725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffefe2fb4a02413ca10f96adc40aa820', 'request_id': 'req-540e40f6-a23c-413d-919b-7590864104cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.968707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffefe2fb4a02413ca10f96adc40aa820', 'request_id': 'req-540e40f6-a23c-413d-919b-7590864104cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.974627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffefe2fb4a02413ca10f96adc40aa820', 'request_id': 'req-540e40f6-a23c-413d-919b-7590864104cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.985467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] The token cache is being invalidate because project ffefe2fb4a02413ca10f96adc40aa820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:36.985690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.986319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-540e40f6-a23c-413d-919b-7590864104cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:36.987834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1030/2065] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:36 2022] DELETE /identity/v3/projects/ffefe2fb4a02413ca10f96adc40aa820 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:36.992714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9cdc-1ad6-4423-94f2-b4a811fd9183 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:36.992999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9cdc-1ad6-4423-94f2-b4a811fd9183 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:36.993271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb9cdc-1ad6-4423-94f2-b4a811fd9183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.013877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5dfb9cdc-1ad6-4423-94f2-b4a811fd9183 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:37.015058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5dfb9cdc-1ad6-4423-94f2-b4a811fd9183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:37.070733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:37.092409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1036/2066] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:37.100154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9086df-9d65-4b9b-9546-78adbc48fc02 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.100475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9086df-9d65-4b9b-9546-78adbc48fc02 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.100776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9086df-9d65-4b9b-9546-78adbc48fc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.120960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1d9086df-9d65-4b9b-9546-78adbc48fc02 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:37.122279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1d9086df-9d65-4b9b-9546-78adbc48fc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:37.131429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1031/2067] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:37.135727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb9cdc-1ad6-4423-94f2-b4a811fd9183 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:37.146211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb9cdc-1ad6-4423-94f2-b4a811fd9183 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:37.151608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:37.152317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.152752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.153090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.153826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:37.154201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:37.155799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:37.174807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1037/2068] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:37.179165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9086df-9d65-4b9b-9546-78adbc48fc02 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:37.189227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9086df-9d65-4b9b-9546-78adbc48fc02 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:37.193814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:37.194405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.194797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.195072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.195700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:37.197611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:37.202035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:37.225259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1032/2069] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:37 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:37.229333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:37.239757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae6439-21c9-485b-a47c-56c877e6117f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:37.294557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:37.295160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.295438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.295712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.296382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:37.298319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:37.320213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1038/2070] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:37 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:37.324432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:37.335143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cafd5-02e5-4f90-8676-5f097d317254 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:37.340138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:37.340772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.341056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.341340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.341979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:37.343570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:37.354169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1033/2071] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:37.357979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:37.368450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514913b-9962-41ca-b9de-9c83b015478e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:37.373223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670459f1-9a92-44a4-9112-aeed30189381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:37.373818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670459f1-9a92-44a4-9112-aeed30189381 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.374094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670459f1-9a92-44a4-9112-aeed30189381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.374362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670459f1-9a92-44a4-9112-aeed30189381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.374997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670459f1-9a92-44a4-9112-aeed30189381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:37.376621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670459f1-9a92-44a4-9112-aeed30189381 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:37.387029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1039/2072] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:37.391835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:37.402173 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86c102-1a16-4407-a62a-1f38adcbeac5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:37.406919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:37.408132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.408436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.408727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] PATH_INFO: `/v3/projects/154f5229567c4023adcbe0069a9b4abb/users/aa5c9d73b6bb4b32b2da9386273690bb/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.423435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=154f5229567c4023adcbe0069a9b4abb, user_id=aa5c9d73b6bb4b32b2da9386273690bb, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:37.426163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:37.442588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1034/2073] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:37 2022] PUT /identity/v3/projects/154f5229567c4023adcbe0069a9b4abb/users/aa5c9d73b6bb4b32b2da9386273690bb/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:57:37.447656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500c51-cd8d-42c2-831a-fbdebbd4008c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.447964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500c51-cd8d-42c2-831a-fbdebbd4008c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.448288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500c51-cd8d-42c2-831a-fbdebbd4008c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.467275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d4500c51-cd8d-42c2-831a-fbdebbd4008c None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:37.468504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d4500c51-cd8d-42c2-831a-fbdebbd4008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:37.545568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1040/2074] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:37.555448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:37.557185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:37.614431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eae6e4-26e6-4b6a-9996-4f78d7d62522 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:37.618138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:37.618737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:37.619011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:37.619279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:37.629258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:37.630784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:37.641330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1035/2075] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:38.362133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf877c6-c431-45a4-89dd-552b84c3b667 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:38.362602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf877c6-c431-45a4-89dd-552b84c3b667 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:38.362746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf877c6-c431-45a4-89dd-552b84c3b667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:38.394752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0cf877c6-c431-45a4-89dd-552b84c3b667 None None] MFA Rules not processed for user `aa5c9d73b6bb4b32b2da9386273690bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:38.395955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0cf877c6-c431-45a4-89dd-552b84c3b667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:38.439259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:38.463423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1041/2076] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:38.474995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:38.476499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:38.539318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed8d36-b6ff-4aea-9417-09329ebfe148 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:38.543075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:38.543674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:38.544079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:38.544215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:38.554335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:38.555699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:38.564995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1036/2077] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:47.313896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5738d-4069-4be4-8329-6a1cf76f8d4c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:47.314335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5738d-4069-4be4-8329-6a1cf76f8d4c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:47.314425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5738d-4069-4be4-8329-6a1cf76f8d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:47.334184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dae5738d-4069-4be4-8329-6a1cf76f8d4c None None] MFA Rules not processed for user `aa5c9d73b6bb4b32b2da9386273690bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:47.335419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dae5738d-4069-4be4-8329-6a1cf76f8d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:47.342175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1042/2078] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:48.560725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=73,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:49.905722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:49.907124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:49.921709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db28d9c-6987-43ca-b812-5aa9f2931636 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:49.927278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:49.927873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:49.928153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:49.928452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] PATH_INFO: `/v3/users/7e81d9321e554466b3f7ec833a6e0215` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:49.929576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e81d9321e554466b3f7ec833a6e0215)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:49.931500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:49.984576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e81d9321e554466b3f7ec833a6e0215'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.001892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e81d9321e554466b3f7ec833a6e0215'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.011393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e81d9321e554466b3f7ec833a6e0215'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.012166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e81d9321e554466b3f7ec833a6e0215'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.012880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e81d9321e554466b3f7ec833a6e0215'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.029371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1037/2079] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:49 2022] DELETE /identity/v3/users/7e81d9321e554466b3f7ec833a6e0215 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:50.033738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:50.034234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:50.034356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:50.064427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:50.066118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:50.173669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1043/2080] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:50.193327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:50.194702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:50.256219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de85e9b-ad5c-4c7a-88a8-638511847189 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:50.259789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:50.260407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:50.260681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:50.260945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:50.269604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:50.271035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:50.280145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1038/2081] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:57:50.541390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:50.542866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:50.559194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29ec57-0198-4d94-bd3c-6169cbd31aec None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:50.564098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:50.564785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:50.565171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:50.565318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] PATH_INFO: `/v3/projects/1fb0fc680f394e6c90d4558766ba577a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:50.566445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb0fc680f394e6c90d4558766ba577a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:50.568412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:50.614090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb0fc680f394e6c90d4558766ba577a', 'request_id': 'req-d7573db2-d697-49af-8dda-746a499af532'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.620175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb0fc680f394e6c90d4558766ba577a', 'request_id': 'req-d7573db2-d697-49af-8dda-746a499af532'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.625590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb0fc680f394e6c90d4558766ba577a', 'request_id': 'req-d7573db2-d697-49af-8dda-746a499af532'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.637892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] The token cache is being invalidate because project 1fb0fc680f394e6c90d4558766ba577a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:50.638175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.638785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.640288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1044/2082] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:50 2022] DELETE /identity/v3/projects/1fb0fc680f394e6c90d4558766ba577a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:50.644183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:50.654191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e341d7f-1404-43b5-9a3b-ff98832d1629 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:50.707655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:50.708761 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:50.709038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:50.709308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] PATH_INFO: `/v3/users/c42e79e9e83e45778716f86e57efee0d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:50.710406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42e79e9e83e45778716f86e57efee0d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:50.712631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:50.728207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:50.759101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42e79e9e83e45778716f86e57efee0d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.771098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42e79e9e83e45778716f86e57efee0d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.777805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42e79e9e83e45778716f86e57efee0d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.778470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42e79e9e83e45778716f86e57efee0d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.779073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42e79e9e83e45778716f86e57efee0d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:50.791788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1039/2083] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:50 2022] DELETE /identity/v3/users/c42e79e9e83e45778716f86e57efee0d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:51.083841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:51.085417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:51.100022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7573db2-d697-49af-8dda-746a499af532 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:51.183306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:51.184072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:51.185109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:51.185109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] PATH_INFO: `/v3/projects/092e63651c624980b16afa1067dd368d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:51.186092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] RBAC: Authorizing `identity:delete_project(project_id=092e63651c624980b16afa1067dd368d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:51.188330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:51.191672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:51.269674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '092e63651c624980b16afa1067dd368d', 'request_id': 'req-45a2b86b-3054-47ec-af22-a77debdfc50d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.278284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '092e63651c624980b16afa1067dd368d', 'request_id': 'req-45a2b86b-3054-47ec-af22-a77debdfc50d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.286464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '092e63651c624980b16afa1067dd368d', 'request_id': 'req-45a2b86b-3054-47ec-af22-a77debdfc50d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.308461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] The token cache is being invalidate because project 092e63651c624980b16afa1067dd368d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:51.309001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.310094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.316365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:51.318119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1045/2084] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:51 2022] DELETE /identity/v3/projects/092e63651c624980b16afa1067dd368d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:51.319278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:51.335101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a1979-3cd5-4d52-a0ca-6dd56f4d24d7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:51.432891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:51.433517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:51.433910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:51.434239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] PATH_INFO: `/v3/users/830bbd32106644cca5856d8ca731b84f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:51.435668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830bbd32106644cca5856d8ca731b84f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:51.437632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:51.499843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830bbd32106644cca5856d8ca731b84f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.520595 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830bbd32106644cca5856d8ca731b84f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.530120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830bbd32106644cca5856d8ca731b84f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.530981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830bbd32106644cca5856d8ca731b84f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.531629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-754e6daa-b786-4aed-a14d-108da057c5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830bbd32106644cca5856d8ca731b84f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:51.546964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1040/2085] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:57:51 2022] DELETE /identity/v3/users/830bbd32106644cca5856d8ca731b84f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:51.957935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:51.959856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:51.993268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2b86b-3054-47ec-af22-a77debdfc50d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.054968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.055513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.055810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.056068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] PATH_INFO: `/v3/projects/2c2d468beb0d4aaf8287568fa65f3130` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.057326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2d468beb0d4aaf8287568fa65f3130)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.059261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.114564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2d468beb0d4aaf8287568fa65f3130', 'request_id': 'req-34e39994-9f58-4be4-9f45-4c06173a8417'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:52.121288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2d468beb0d4aaf8287568fa65f3130', 'request_id': 'req-34e39994-9f58-4be4-9f45-4c06173a8417'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:52.127379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2d468beb0d4aaf8287568fa65f3130', 'request_id': 'req-34e39994-9f58-4be4-9f45-4c06173a8417'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:52.139836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] The token cache is being invalidate because project 2c2d468beb0d4aaf8287568fa65f3130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:57:52.140139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:52.140800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-34e39994-9f58-4be4-9f45-4c06173a8417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:57:52.142379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1046/2086] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:57:51 2022] DELETE /identity/v3/projects/2c2d468beb0d4aaf8287568fa65f3130 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:52.147706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e151468-7a55-4ba3-9be5-eafcc7e3356a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.147997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e151468-7a55-4ba3-9be5-eafcc7e3356a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.148386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e151468-7a55-4ba3-9be5-eafcc7e3356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.169071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5e151468-7a55-4ba3-9be5-eafcc7e3356a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:52.170314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5e151468-7a55-4ba3-9be5-eafcc7e3356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:52.179740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:52.237138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1041/2087] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:52.244869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948e9a-e1c2-48ed-9e12-002a5b4af0ee None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.245254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948e9a-e1c2-48ed-9e12-002a5b4af0ee None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.245599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948e9a-e1c2-48ed-9e12-002a5b4af0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.267252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0d948e9a-e1c2-48ed-9e12-002a5b4af0ee None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:52.268666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0d948e9a-e1c2-48ed-9e12-002a5b4af0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:52.277893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1047/2088] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:57:52.281923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e151468-7a55-4ba3-9be5-eafcc7e3356a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:52.295004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e151468-7a55-4ba3-9be5-eafcc7e3356a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.303305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.303913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.304190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.304500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.305207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:57:52.305456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.307728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.324234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1042/2089] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:57:52.329007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948e9a-e1c2-48ed-9e12-002a5b4af0ee None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:52.338647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948e9a-e1c2-48ed-9e12-002a5b4af0ee None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.343363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.343953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.344311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.344592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.345233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.347139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.359627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:57:52.383710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1048/2090] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:57:52 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:57:52.383937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:52.394161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b91fd-f258-4e29-8512-db7e800ff352 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.457316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.458010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.458925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.459256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.460082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.462359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.485417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1043/2091] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:57:52 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:57:52.489407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:52.500069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c6d25-e829-450b-984c-0d9bec79d4c0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.505544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.506146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.506420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.506688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.507328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.509011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.519080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1049/2092] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:52.523851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:52.534278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca15f5-f6c1-46af-b7eb-703a21d5867c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.539005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279635e-33af-40e8-857b-e0ec1fad002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.539604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279635e-33af-40e8-857b-e0ec1fad002d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.539881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279635e-33af-40e8-857b-e0ec1fad002d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.540156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279635e-33af-40e8-857b-e0ec1fad002d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.540839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279635e-33af-40e8-857b-e0ec1fad002d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.542414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279635e-33af-40e8-857b-e0ec1fad002d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.553464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1044/2093] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:57:52.557700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:52.568608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d51d8-be80-4a8f-b822-a33ccc05cdca admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.573419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.574078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.574500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.574781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] PATH_INFO: `/v3/projects/bba6de9ec9a341d79c58a2a45219be52/users/1cbc3198832c4f6796c073817360ff72/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.592119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba6de9ec9a341d79c58a2a45219be52, user_id=1cbc3198832c4f6796c073817360ff72, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.595134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.618891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1050/2094] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:57:52 2022] PUT /identity/v3/projects/bba6de9ec9a341d79c58a2a45219be52/users/1cbc3198832c4f6796c073817360ff72/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:57:52.620592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.620869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.621227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.659727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] MFA Rules not processed for user `1cbc3198832c4f6796c073817360ff72`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:57:52.661211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:52.732799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1045/2095] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:57:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:57:52.745092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:57:52.746847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:57:52.809901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a50ef7-9501-4b3f-960a-58dd0da5c86e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:57:52.813772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f80d3f-f345-4244-a8cf-a3f88ebb1052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:57:52.814380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f80d3f-f345-4244-a8cf-a3f88ebb1052 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:57:52.814657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f80d3f-f345-4244-a8cf-a3f88ebb1052 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:57:52.814926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f80d3f-f345-4244-a8cf-a3f88ebb1052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:57:52.823876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f80d3f-f345-4244-a8cf-a3f88ebb1052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:57:52.825330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f80d3f-f345-4244-a8cf-a3f88ebb1052 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:57:52.834159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1051/2096] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:57:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:02.729978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:02.829995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:07.880009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:07.881630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:07.958732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fafa30-74bd-4e35-99c7-baa24887bd3c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:07.965013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:07.965757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:07.966142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:07.966518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] PATH_INFO: `/v3/users/b0954439d74a46ac914bcc7b669ac360` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:07.967773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0954439d74a46ac914bcc7b669ac360)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:07.969954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:08.048039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0954439d74a46ac914bcc7b669ac360'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:08.063139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0954439d74a46ac914bcc7b669ac360'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:08.074094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0954439d74a46ac914bcc7b669ac360'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:08.074877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0954439d74a46ac914bcc7b669ac360'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:08.075498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0954439d74a46ac914bcc7b669ac360'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:08.091572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1046/2097] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:58:07 2022] DELETE /identity/v3/users/b0954439d74a46ac914bcc7b669ac360 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:58:08.096160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:08.096663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:08.097071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:08.118345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:08.119635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:08.198032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1052/2098] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:58:08.211301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:08.213083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:08.314606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea816b-2fa9-4739-86d9-6c4c3542b6bf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:08.321082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a16c2-1f21-4183-87b7-0de2c098b150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:08.321990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a16c2-1f21-4183-87b7-0de2c098b150 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:08.322406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a16c2-1f21-4183-87b7-0de2c098b150 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:08.322810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a16c2-1f21-4183-87b7-0de2c098b150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:08.342739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a16c2-1f21-4183-87b7-0de2c098b150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:08.345787 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a16c2-1f21-4183-87b7-0de2c098b150 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:08.379702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1047/2099] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:08.917209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:08.918642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:08.930963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75255-cd1e-4dd8-adab-dcc14fd196be None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:08.954928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:08.955585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:08.955852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:08.956125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] PATH_INFO: `/v3/projects/77999249e1924fb5a6346bb22ea17a78` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:08.957457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] RBAC: Authorizing `identity:delete_project(project_id=77999249e1924fb5a6346bb22ea17a78)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:08.960173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.015306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77999249e1924fb5a6346bb22ea17a78', 'request_id': 'req-294531ed-5331-43c4-bc70-66b71e1a333c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:09.023479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77999249e1924fb5a6346bb22ea17a78', 'request_id': 'req-294531ed-5331-43c4-bc70-66b71e1a333c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:09.030082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77999249e1924fb5a6346bb22ea17a78', 'request_id': 'req-294531ed-5331-43c4-bc70-66b71e1a333c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:09.047968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] The token cache is being invalidate because project 77999249e1924fb5a6346bb22ea17a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:58:09.048372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:09.050487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-294531ed-5331-43c4-bc70-66b71e1a333c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:09.052269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1053/2100] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:58:08 2022] DELETE /identity/v3/projects/77999249e1924fb5a6346bb22ea17a78 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:58:09.057259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c181ec-e763-4307-8d68-36cabf69bd29 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.057667 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c181ec-e763-4307-8d68-36cabf69bd29 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.057845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c181ec-e763-4307-8d68-36cabf69bd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.082197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-82c181ec-e763-4307-8d68-36cabf69bd29 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:09.083506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-82c181ec-e763-4307-8d68-36cabf69bd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:09.105425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:09.178687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1048/2101] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:09.179318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1aea-3987-4231-9122-7c0c6ac0e90f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.179791 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1aea-3987-4231-9122-7c0c6ac0e90f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.180023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1aea-3987-4231-9122-7c0c6ac0e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.201110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f26b1aea-3987-4231-9122-7c0c6ac0e90f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:09.202932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f26b1aea-3987-4231-9122-7c0c6ac0e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:09.212387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1054/2102] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:58:09.216306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c181ec-e763-4307-8d68-36cabf69bd29 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:09.226940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c181ec-e763-4307-8d68-36cabf69bd29 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:09.232861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:09.233772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.234233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.235303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.236546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:58:09.236861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:09.238473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.256219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1049/2103] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:58:09.260939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b1aea-3987-4231-9122-7c0c6ac0e90f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:09.272161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b1aea-3987-4231-9122-7c0c6ac0e90f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:09.278484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:09.279539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.280299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.280574 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.281240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:09.283238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.325195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1055/2104] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:58:09 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:58:09.335629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:09.351824 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27a550-c1ac-4c6c-aec3-97f54bebce6e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:09.415568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:09.416204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.416565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.416876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.417559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:09.419518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.443281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1050/2105] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:58:09 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:58:09.447746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:09.458934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0793c0f-98c2-41a1-98d4-0f0263291c6e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:09.463625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:09.464222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.464524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.464796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.465442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:09.467133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.477280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1056/2106] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:58:09.481306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:09.491673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6f845-5674-41e4-8b36-773c7a405d29 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:09.496371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239e47b-1539-4368-b0c6-a0cbc46bb7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:09.496966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e47b-1539-4368-b0c6-a0cbc46bb7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.497244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e47b-1539-4368-b0c6-a0cbc46bb7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.497516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239e47b-1539-4368-b0c6-a0cbc46bb7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.498151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239e47b-1539-4368-b0c6-a0cbc46bb7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:09.499735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239e47b-1539-4368-b0c6-a0cbc46bb7f9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.510954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1051/2107] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:58:09.514881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:09.524623 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8e763-e6f1-4f63-aeed-7c429698df1d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:09.529189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:09.529822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.530099 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.530396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] PATH_INFO: `/v3/projects/e373d805554c445dbeb421d858c27bd7/users/16f67990056b4057a6f1a9a3f96ac1e6/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.544463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e373d805554c445dbeb421d858c27bd7, user_id=16f67990056b4057a6f1a9a3f96ac1e6, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:09.547138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.561042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1057/2108] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:58:09 2022] PUT /identity/v3/projects/e373d805554c445dbeb421d858c27bd7/users/16f67990056b4057a6f1a9a3f96ac1e6/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:58:09.571678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.571969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.572272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.598857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] MFA Rules not processed for user `16f67990056b4057a6f1a9a3f96ac1e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:09.600055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:09.671800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1052/2109] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:09.683878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:09.685574 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:09.706656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:09.744006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a804-2939-4cb3-92ba-58256cbfa18a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:09.747610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f44463-2bef-4047-93a1-98b0ff7c5578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:09.748210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44463-2bef-4047-93a1-98b0ff7c5578 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:09.748511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44463-2bef-4047-93a1-98b0ff7c5578 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:09.748780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44463-2bef-4047-93a1-98b0ff7c5578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:09.757342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f44463-2bef-4047-93a1-98b0ff7c5578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:09.758757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f44463-2bef-4047-93a1-98b0ff7c5578 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:09.767556 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1058/2110] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:11.337739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:11.339305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:11.361090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:11.406934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25335587-fe1a-450f-8646-26999b7c8c29 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:11.411393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:11.412568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:11.412843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:11.413112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] PATH_INFO: `/v3/users/1cbc3198832c4f6796c073817360ff72` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:11.414217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cbc3198832c4f6796c073817360ff72)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:11.416114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:11.461348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cbc3198832c4f6796c073817360ff72'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:11.472998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cbc3198832c4f6796c073817360ff72'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:11.480413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cbc3198832c4f6796c073817360ff72'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:11.481077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cbc3198832c4f6796c073817360ff72'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:11.481672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cbc3198832c4f6796c073817360ff72'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:11.495149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1053/2111] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:58:11 2022] DELETE /identity/v3/users/1cbc3198832c4f6796c073817360ff72 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:58:11.499589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:11.500039 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:11.500130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:11.519837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:11.521040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:11.585635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1059/2112] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:11.595256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:11.596701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:11.658254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508f31-cd14-41e9-9092-632d3edae518 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:11.662908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cabc6-df57-46a5-aee8-abd877740b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:11.663526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cabc6-df57-46a5-aee8-abd877740b9e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:11.663831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cabc6-df57-46a5-aee8-abd877740b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:11.664132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cabc6-df57-46a5-aee8-abd877740b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:11.674468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cabc6-df57-46a5-aee8-abd877740b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:11.676054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cabc6-df57-46a5-aee8-abd877740b9e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:11.686125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1054/2113] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:12.033140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.034568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.050382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479308d-aeac-42e0-8f9e-8dffa383ae14 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.056366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.057040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.057234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.057507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] PATH_INFO: `/v3/projects/bba6de9ec9a341d79c58a2a45219be52` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.058873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba6de9ec9a341d79c58a2a45219be52)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.060842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.112068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba6de9ec9a341d79c58a2a45219be52', 'request_id': 'req-9110eeb0-2226-427e-b3e1-17793c0b092b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.118372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba6de9ec9a341d79c58a2a45219be52', 'request_id': 'req-9110eeb0-2226-427e-b3e1-17793c0b092b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.125292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba6de9ec9a341d79c58a2a45219be52', 'request_id': 'req-9110eeb0-2226-427e-b3e1-17793c0b092b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.139741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] The token cache is being invalidate because project bba6de9ec9a341d79c58a2a45219be52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:58:12.140120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.140933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9110eeb0-2226-427e-b3e1-17793c0b092b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.142792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1060/2114] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:58:12 2022] DELETE /identity/v3/projects/bba6de9ec9a341d79c58a2a45219be52 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:58:12.148969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2aea6-4333-4906-8589-c5a700e8ffac None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.149350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2aea6-4333-4906-8589-c5a700e8ffac None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.149668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2aea6-4333-4906-8589-c5a700e8ffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.171228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-67b2aea6-4333-4906-8589-c5a700e8ffac None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:12.172709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-67b2aea6-4333-4906-8589-c5a700e8ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.217356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:12.246253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1055/2115] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:12.253953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d6263-acf4-48e9-a237-312b547375d9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.254358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d6263-acf4-48e9-a237-312b547375d9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.254702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d6263-acf4-48e9-a237-312b547375d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.277465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-767d6263-acf4-48e9-a237-312b547375d9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:12.278588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-767d6263-acf4-48e9-a237-312b547375d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.286857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1061/2116] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:12.291004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2aea6-4333-4906-8589-c5a700e8ffac None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.301710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2aea6-4333-4906-8589-c5a700e8ffac None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.306963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.307599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.307882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.308158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.308840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:58:12.309145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.310728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.323010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1056/2117] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:58:12.328417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d6263-acf4-48e9-a237-312b547375d9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.338675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d6263-acf4-48e9-a237-312b547375d9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.343578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.344182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.344489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.344768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.348426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.348426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.350131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.352608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.368443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8a910-a84b-40ee-9ee9-b1b643c4b275 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.373842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.374563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.374911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.375219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] PATH_INFO: `/v3/users/16f67990056b4057a6f1a9a3f96ac1e6` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.376577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f67990056b4057a6f1a9a3f96ac1e6)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.377151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1062/2118] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:58:12 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:58:12.378933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.385389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.411626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:12.432812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f67990056b4057a6f1a9a3f96ac1e6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.446756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f67990056b4057a6f1a9a3f96ac1e6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.454587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f67990056b4057a6f1a9a3f96ac1e6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.455408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f67990056b4057a6f1a9a3f96ac1e6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.456133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3f47f849-f629-4c5f-a0dd-650f57d6e2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f67990056b4057a6f1a9a3f96ac1e6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:12.464157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6052-c3a8-488c-9dee-3cc3a2069f0d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.470114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.470794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1057/2119] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:58:12 2022] DELETE /identity/v3/users/16f67990056b4057a6f1a9a3f96ac1e6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:58:12.472820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.473147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.474634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.474940 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.475209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.476543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.477281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.479468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.500639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:12.502112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.503244 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1063/2120] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:58:12 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:58:12.507367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.509098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.521428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3f412-6b0b-489c-9868-bf67f9c6403d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.569662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1058/2121] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:12.579360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.580765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.583996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.584652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.584918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.585185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.585825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.587409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.599908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1064/2122] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:58:12.600875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.610457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ddfb1-5844-40da-90a7-fb5e5b33bdd0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.615779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.616388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.616701 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.616977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.617633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.619248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.623563 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:12.630499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1065/2123] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:58:12.635803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.641093 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a89e18-1c53-4d9f-a524-a9258f26867a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.645744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c9817-56f2-4bcb-a46c-b89894022940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.646597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c9817-56f2-4bcb-a46c-b89894022940 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.646875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c9817-56f2-4bcb-a46c-b89894022940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.647149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c9817-56f2-4bcb-a46c-b89894022940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.647253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3234-8ad2-45ed-bd64-06462483d7e7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.652947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.653601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.653909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.654172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] PATH_INFO: `/v3/projects/4d07d38e84224c9282dea262e712e4e3/users/0646d81951fa4353bdfb59048b498652/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.657811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c9817-56f2-4bcb-a46c-b89894022940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.659306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c9817-56f2-4bcb-a46c-b89894022940 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.669539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d07d38e84224c9282dea262e712e4e3, user_id=0646d81951fa4353bdfb59048b498652, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.669886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1059/2124] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:58:12.672157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.686135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1066/2125] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:58:12 2022] PUT /identity/v3/projects/4d07d38e84224c9282dea262e712e4e3/users/0646d81951fa4353bdfb59048b498652/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:58:12.691183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.691468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.691742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.710965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:12.712214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.777368 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1060/2126] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:12.784050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.785861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.851336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99099df8-3b6b-462a-ac87-d4b599c41e32 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.855062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d18f02-1cb6-4390-bf2a-85ca73d6b1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.855666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d18f02-1cb6-4390-bf2a-85ca73d6b1ab service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.855940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d18f02-1cb6-4390-bf2a-85ca73d6b1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.856207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d18f02-1cb6-4390-bf2a-85ca73d6b1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.865878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d18f02-1cb6-4390-bf2a-85ca73d6b1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.867306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d18f02-1cb6-4390-bf2a-85ca73d6b1ab service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:12.877057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1067/2127] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:12.945429 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:12.946971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:12.967350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fe355-a940-4d1b-bb95-f4172145ca0e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:12.973860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:12.975216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:12.975558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:12.975864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] PATH_INFO: `/v3/projects/e373d805554c445dbeb421d858c27bd7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:12.985457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e373d805554c445dbeb421d858c27bd7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:12.987363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:13.046397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e373d805554c445dbeb421d858c27bd7', 'request_id': 'req-4069e052-62b0-4368-8977-6c8237fcff74'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:13.054694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e373d805554c445dbeb421d858c27bd7', 'request_id': 'req-4069e052-62b0-4368-8977-6c8237fcff74'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:13.062018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e373d805554c445dbeb421d858c27bd7', 'request_id': 'req-4069e052-62b0-4368-8977-6c8237fcff74'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:13.074710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] The token cache is being invalidate because project e373d805554c445dbeb421d858c27bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:58:13.075192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:13.076066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:58:13.077610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1061/2128] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:58:12 2022] DELETE /identity/v3/projects/e373d805554c445dbeb421d858c27bd7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:58:13.571392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5ded-76ea-495c-a165-980f5136ec2a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:13.571951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5ded-76ea-495c-a165-980f5136ec2a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:13.572068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5ded-76ea-495c-a165-980f5136ec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:13.594827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:13.607949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e45e5ded-76ea-495c-a165-980f5136ec2a None None] MFA Rules not processed for user `0646d81951fa4353bdfb59048b498652`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:13.609802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e45e5ded-76ea-495c-a165-980f5136ec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:13.687088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1068/2129] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:13.699514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:13.701453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:13.762072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069e052-62b0-4368-8977-6c8237fcff74 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:13.765616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056792e9-f556-4a40-9ad2-23f8e208f39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:13.766215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056792e9-f556-4a40-9ad2-23f8e208f39c service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:13.766513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056792e9-f556-4a40-9ad2-23f8e208f39c service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:13.766782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056792e9-f556-4a40-9ad2-23f8e208f39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:13.775506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056792e9-f556-4a40-9ad2-23f8e208f39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:13.776953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056792e9-f556-4a40-9ad2-23f8e208f39c service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:13.785716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1062/2130] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:15.212974 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:15.213511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:15.213650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:15.236080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] MFA Rules not processed for user `0646d81951fa4353bdfb59048b498652`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:15.237486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:15.245941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1069/2131] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:17.196660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832b627-c670-447f-ba92-5626c49b76af None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:17.197192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832b627-c670-447f-ba92-5626c49b76af None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:17.197635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832b627-c670-447f-ba92-5626c49b76af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:17.217016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5832b627-c670-447f-ba92-5626c49b76af None None] MFA Rules not processed for user `aa5c9d73b6bb4b32b2da9386273690bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:17.218551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5832b627-c670-447f-ba92-5626c49b76af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:17.222731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:17.287602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1063/2132] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:58:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:25.232698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=30 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:27.274073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=21 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:27.793294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:27.794863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:27.852426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f7652-2be1-4aef-9b5d-88310060e236 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:27.855899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8065513-e24d-4849-aa0e-26c83fb73d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:27.856534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8065513-e24d-4849-aa0e-26c83fb73d15 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:27.856810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8065513-e24d-4849-aa0e-26c83fb73d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:27.857079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8065513-e24d-4849-aa0e-26c83fb73d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:27.858781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b8065513-e24d-4849-aa0e-26c83fb73d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:27.921212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8065513-e24d-4849-aa0e-26c83fb73d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:27.922679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8065513-e24d-4849-aa0e-26c83fb73d15 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:27.932161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1070/2133] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:37.927794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=50 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:44.859407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9e423-c81c-4a6b-a677-005e544928d9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:44.859805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9e423-c81c-4a6b-a677-005e544928d9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:44.859984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9e423-c81c-4a6b-a677-005e544928d9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:44.861376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1064/2134] 10.222.0.50 () {54 vars in 961 bytes} [Tue Aug 30 01:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:58:44.866873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca71a-c9bc-4354-81f1-b91bc4af69df None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:44.867306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca71a-c9bc-4354-81f1-b91bc4af69df None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:44.867402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca71a-c9bc-4354-81f1-b91bc4af69df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:44.887248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-225ca71a-c9bc-4354-81f1-b91bc4af69df None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:44.888435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-225ca71a-c9bc-4354-81f1-b91bc4af69df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:44.966678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1071/2135] 10.222.0.50 () {58 vars in 1058 bytes} [Tue Aug 30 01:58:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:44.974209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9e423-c81c-4a6b-a677-005e544928d9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:44.996623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9e423-c81c-4a6b-a677-005e544928d9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:45.001915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:45.002553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:45.002835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:45.003112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:45.005120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:45.016882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:45.018475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:45.028600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1065/2136] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:58:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:58:54.953686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=29 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:55.024016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:58:56.705935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1709-8808-43f4-81f4-7888f7d951d3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:56.706429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1709-8808-43f4-81f4-7888f7d951d3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:56.706534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1709-8808-43f4-81f4-7888f7d951d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:56.726752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-257f1709-8808-43f4-81f4-7888f7d951d3 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:58:56.727933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-257f1709-8808-43f4-81f4-7888f7d951d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:56.782947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1072/2137] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:58:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:58:56.823480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:58:56.824884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:58:56.893057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a056e0-bed5-4f1f-8bd6-b129f2e828c1 service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:58:56.898397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:58:56.898991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:58:56.899266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:58:56.899536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:58:56.908089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:58:56.909694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:58:56.918345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1066/2138] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 01:58:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:59:06.776152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=25 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:59:06.915064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=32 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:59:13.819523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27652-4733-464d-9d08-5604caf1df6a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:13.820109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27652-4733-464d-9d08-5604caf1df6a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:13.820109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27652-4733-464d-9d08-5604caf1df6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:13.838480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-def27652-4733-464d-9d08-5604caf1df6a None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:59:13.839636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-def27652-4733-464d-9d08-5604caf1df6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:13.846585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1073/2139] 10.222.0.50 () {58 vars in 967 bytes} [Tue Aug 30 01:59:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:59:13.859651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:13.870437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c4e7c-25a5-44c8-80d4-963dc17b86fe service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:13.875414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:13.876003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:13.876301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:13.876573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:13.885475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:13.887092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:13.895122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1067/2140] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 01:59:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:59:15.609152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26a48-38ff-4716-a7d8-e972e29382d1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:15.610070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26a48-38ff-4716-a7d8-e972e29382d1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:15.610416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26a48-38ff-4716-a7d8-e972e29382d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:15.633212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e6e26a48-38ff-4716-a7d8-e972e29382d1 None None] MFA Rules not processed for user `86f364f03c7d4360b4a75ed194846ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:59:15.633212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e6e26a48-38ff-4716-a7d8-e972e29382d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:15.689270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1074/2141] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:59:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:59:15.977379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:15.979004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:16.048819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3cefa-d144-4805-92b4-27af997de8a8 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.053462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.054090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.054364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.054630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] PATH_INFO: `/v3/users/aa5c9d73b6bb4b32b2da9386273690bb` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.055748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5c9d73b6bb4b32b2da9386273690bb)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.057691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.106574 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5c9d73b6bb4b32b2da9386273690bb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.118711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5c9d73b6bb4b32b2da9386273690bb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.126445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5c9d73b6bb4b32b2da9386273690bb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.127287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5c9d73b6bb4b32b2da9386273690bb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.127926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-02b87175-fa85-4735-a000-49c33a745660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5c9d73b6bb4b32b2da9386273690bb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.141439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1068/2142] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 01:59:15 2022] DELETE /identity/v3/users/aa5c9d73b6bb4b32b2da9386273690bb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:59:16.319950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e26a48-38ff-4716-a7d8-e972e29382d1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:16.334804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e26a48-38ff-4716-a7d8-e972e29382d1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.400119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.400773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.401248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.401520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] PATH_INFO: `/v3/projects/154f5229567c4023adcbe0069a9b4abb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.402643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=154f5229567c4023adcbe0069a9b4abb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.404566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.459947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154f5229567c4023adcbe0069a9b4abb', 'request_id': 'req-38f08ea7-4b93-4195-a9b2-77644db86181'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.466991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154f5229567c4023adcbe0069a9b4abb', 'request_id': 'req-38f08ea7-4b93-4195-a9b2-77644db86181'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.478323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154f5229567c4023adcbe0069a9b4abb', 'request_id': 'req-38f08ea7-4b93-4195-a9b2-77644db86181'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.492390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] The token cache is being invalidate because project 154f5229567c4023adcbe0069a9b4abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:59:16.492676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.493296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-38f08ea7-4b93-4195-a9b2-77644db86181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:59:16.495063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1075/2143] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 01:59:16 2022] DELETE /identity/v3/projects/154f5229567c4023adcbe0069a9b4abb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:59:16.499719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21bd4d-29ab-4aa3-9591-1df5d5109b19 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.500031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21bd4d-29ab-4aa3-9591-1df5d5109b19 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.500379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21bd4d-29ab-4aa3-9591-1df5d5109b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.522569 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5a21bd4d-29ab-4aa3-9591-1df5d5109b19 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:59:16.523759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5a21bd4d-29ab-4aa3-9591-1df5d5109b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:16.586819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1069/2144] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:59:16.594365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fe438-a0b2-4559-ae21-cac8b25317eb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.594657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fe438-a0b2-4559-ae21-cac8b25317eb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.594935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fe438-a0b2-4559-ae21-cac8b25317eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.614732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2f1fe438-a0b2-4559-ae21-cac8b25317eb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:59:16.615862 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2f1fe438-a0b2-4559-ae21-cac8b25317eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:16.627056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1076/2145] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:59:16.628559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21bd4d-29ab-4aa3-9591-1df5d5109b19 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:16.641647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21bd4d-29ab-4aa3-9591-1df5d5109b19 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.647744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.648406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.648658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.649134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.650834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:59:16.651347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.654088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.655989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:59:16.672416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1070/2146] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 01:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:59:16.672863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fe438-a0b2-4559-ae21-cac8b25317eb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:16.682230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fe438-a0b2-4559-ae21-cac8b25317eb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.687432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.688033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.688336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.688777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.689584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.691643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.719003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:59:16.733973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:16.734664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1077/2147] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 01:59:16 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:59:16.744510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ed3a9-babd-44ae-a6f7-41cf906253e2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.816972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.818549 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.818889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.819176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.819911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.821872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.844551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1071/2148] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 01:59:16 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:59:16.848867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:16.859342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48942be-3135-40f3-b822-52c882c5570c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.864085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.865306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.865527 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.865799 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.866435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.868029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.878111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1078/2149] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:59:16.882978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:16.893133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9ecfa-0894-488d-8560-1b14debca90e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.897801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb302b3-dc08-4a26-b245-e6c6c1b4c74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.898394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb302b3-dc08-4a26-b245-e6c6c1b4c74a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.898675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb302b3-dc08-4a26-b245-e6c6c1b4c74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.898952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb302b3-dc08-4a26-b245-e6c6c1b4c74a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.899587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb302b3-dc08-4a26-b245-e6c6c1b4c74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.901207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb302b3-dc08-4a26-b245-e6c6c1b4c74a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.911968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1072/2150] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 01:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:59:16.916102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:16.926134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978a5a4-63ad-425c-a577-230bb81a5255 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:16.931325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:16.932025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.932342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.932625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] PATH_INFO: `/v3/projects/ce7b817e87df4b87b7b113e3bb8fc7db/users/27d0e3a644e94f5eb453acda5fd5ad74/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:16.947757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7b817e87df4b87b7b113e3bb8fc7db, user_id=27d0e3a644e94f5eb453acda5fd5ad74, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:16.950442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:16.962950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1079/2151] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 01:59:16 2022] PUT /identity/v3/projects/ce7b817e87df4b87b7b113e3bb8fc7db/users/27d0e3a644e94f5eb453acda5fd5ad74/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:59:16.973838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae610-8919-4b04-8fa2-5ee57220847e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:16.974147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae610-8919-4b04-8fa2-5ee57220847e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:16.974422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae610-8919-4b04-8fa2-5ee57220847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:17.001391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-562ae610-8919-4b04-8fa2-5ee57220847e None None] MFA Rules not processed for user `27d0e3a644e94f5eb453acda5fd5ad74`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:59:17.002586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-562ae610-8919-4b04-8fa2-5ee57220847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:17.064739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1073/2152] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:59:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:59:17.076755 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:17.078423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:17.134844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f18aeb-f2d4-4d42-bbd6-47a05eca3763 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:17.138465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:17.139064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:17.139343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:17.139618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:17.148005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:17.149469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:17.158154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1080/2153] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 01:59:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:59:27.061826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:59:27.154164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:59:47.384768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058daea4-ac87-4adb-a48b-b0bef7e3e2ae None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:47.385247 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058daea4-ac87-4adb-a48b-b0bef7e3e2ae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:47.385430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058daea4-ac87-4adb-a48b-b0bef7e3e2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:47.404840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-058daea4-ac87-4adb-a48b-b0bef7e3e2ae None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:59:47.406220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-058daea4-ac87-4adb-a48b-b0bef7e3e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:47.460383 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1074/2154] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 01:59:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:59:47.472324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:59:47.473732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:47.544549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75703f98-f909-4a8a-b08b-2ea405521d93 service glance] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:59:47.549478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:59:47.550079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:47.550352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:47.550619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:47.559314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:59:47.560969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:59:47.573253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1081/2155] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 01:59:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:59:55.715655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a8573-2b12-439d-8a20-f0a08e048605 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:59:55.716499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a8573-2b12-439d-8a20-f0a08e048605 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:59:55.716784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a8573-2b12-439d-8a20-f0a08e048605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:59:55.754407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e60a8573-2b12-439d-8a20-f0a08e048605 None None] MFA Rules not processed for user `a06de37941264b979a3bd5aaf84ec73c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:59:55.756015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e60a8573-2b12-439d-8a20-f0a08e048605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:59:55.828078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1075/2156] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 01:59:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:59:57.569303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=32 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:05.820859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=50 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:07.872779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:07.934432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f52d6-a31e-4cd3-81d5-bb244c0782cc service swift] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:07.938994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:07.939617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:07.939891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:07.940159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] PATH_INFO: `/v3/users/27d0e3a644e94f5eb453acda5fd5ad74` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:07.941340 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d0e3a644e94f5eb453acda5fd5ad74)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:07.943249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:07.990368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d0e3a644e94f5eb453acda5fd5ad74'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:07.997055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d0e3a644e94f5eb453acda5fd5ad74'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.003819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d0e3a644e94f5eb453acda5fd5ad74'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.004472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d0e3a644e94f5eb453acda5fd5ad74'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.016209 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d0e3a644e94f5eb453acda5fd5ad74'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.021975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1082/2157] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:00:07 2022] DELETE /identity/v3/users/27d0e3a644e94f5eb453acda5fd5ad74 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:08.027258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.027700 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.027792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.047861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:08.049077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:08.105797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1076/2158] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:08.115812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:08.117241 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:08.174439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741d81e-960c-4a3a-9af0-3b29b56bc9b1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:08.177985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce52f2-5d99-4a4a-ac83-c2e392dc952f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:08.178579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce52f2-5d99-4a4a-ac83-c2e392dc952f service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.178856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce52f2-5d99-4a4a-ac83-c2e392dc952f service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.179125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce52f2-5d99-4a4a-ac83-c2e392dc952f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.187791 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce52f2-5d99-4a4a-ac83-c2e392dc952f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:08.189247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce52f2-5d99-4a4a-ac83-c2e392dc952f service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:08.200362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1083/2159] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:08.543987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:08.545485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:08.558045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3863fc7-376a-4b94-85c4-3b4fb07bd2f1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:08.562848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:08.563479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.563760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.564037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] PATH_INFO: `/v3/projects/ce7b817e87df4b87b7b113e3bb8fc7db` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.565461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7b817e87df4b87b7b113e3bb8fc7db)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:08.567085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:08.613213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7b817e87df4b87b7b113e3bb8fc7db', 'request_id': 'req-79aaa5e8-bcc3-4c25-b82f-3c5552276632'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.621284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7b817e87df4b87b7b113e3bb8fc7db', 'request_id': 'req-79aaa5e8-bcc3-4c25-b82f-3c5552276632'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.626850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7b817e87df4b87b7b113e3bb8fc7db', 'request_id': 'req-79aaa5e8-bcc3-4c25-b82f-3c5552276632'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.638599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] The token cache is being invalidate because project ce7b817e87df4b87b7b113e3bb8fc7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:00:08.638887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.639519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-79aaa5e8-bcc3-4c25-b82f-3c5552276632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:08.641037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1077/2160] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:00:08 2022] DELETE /identity/v3/projects/ce7b817e87df4b87b7b113e3bb8fc7db => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:08.645876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da02f4-6376-4bea-8fec-354f45ea69ab None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.646161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da02f4-6376-4bea-8fec-354f45ea69ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.646435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da02f4-6376-4bea-8fec-354f45ea69ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.665265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b6da02f4-6376-4bea-8fec-354f45ea69ab None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:08.666434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b6da02f4-6376-4bea-8fec-354f45ea69ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:08.683114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:08.727541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1084/2161] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:08.735610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03e0b1-c5bf-4d29-9886-8290875d47ab None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.735971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03e0b1-c5bf-4d29-9886-8290875d47ab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.736292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03e0b1-c5bf-4d29-9886-8290875d47ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.756834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ae03e0b1-c5bf-4d29-9886-8290875d47ab None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:08.758011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ae03e0b1-c5bf-4d29-9886-8290875d47ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:08.766756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1078/2162] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:08.770792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da02f4-6376-4bea-8fec-354f45ea69ab None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:08.780595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da02f4-6376-4bea-8fec-354f45ea69ab None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:08.785200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:08.785800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.786076 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.786352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.786996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:00:08.787301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:08.788949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:08.800711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1085/2163] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:00:08.804805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03e0b1-c5bf-4d29-9886-8290875d47ab None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:08.814370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03e0b1-c5bf-4d29-9886-8290875d47ab None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:08.823617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:08.824206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.824511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.824781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.825399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:08.827259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:08.853635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1079/2164] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:00:08 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:00:08.857750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:08.867537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99a136-19f7-4cdf-b99b-acbd2ea84088 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:08.924448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:08.925051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.925317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.925590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.926254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:08.928207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:08.952117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1086/2165] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:00:08 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:00:08.953303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:08.963669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c4917-74fc-46a7-b751-099ca9869630 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:08.968821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:08.969454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:08.969734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:08.970007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:08.970680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:08.972316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:08.989384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1080/2166] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:00:08.990010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:09.000932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a777f-1952-4a75-8a21-0794884dd6d9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:09.005876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dbbd1-a904-4aed-884b-b971c6b56f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:09.006471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbbd1-a904-4aed-884b-b971c6b56f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.006747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbbd1-a904-4aed-884b-b971c6b56f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.007020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dbbd1-a904-4aed-884b-b971c6b56f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.007660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dbbd1-a904-4aed-884b-b971c6b56f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:09.009304 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dbbd1-a904-4aed-884b-b971c6b56f83 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:09.020896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1087/2167] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:00:09.025124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:09.035175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200882-6c45-48f7-8e5a-ac2b0e8313d8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:09.039776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:09.040988 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.041266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.041579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] PATH_INFO: `/v3/projects/12181383c83046ef96403279e1d25147/users/0a813031f734439f91679e893f7b265f/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.057212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12181383c83046ef96403279e1d25147, user_id=0a813031f734439f91679e893f7b265f, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:09.059855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:09.073763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1081/2168] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:00:09 2022] PUT /identity/v3/projects/12181383c83046ef96403279e1d25147/users/0a813031f734439f91679e893f7b265f/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:09.084853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466df4de-89c7-4431-8ed8-fe98cc134d99 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.085176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466df4de-89c7-4431-8ed8-fe98cc134d99 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.085445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466df4de-89c7-4431-8ed8-fe98cc134d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.113257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-466df4de-89c7-4431-8ed8-fe98cc134d99 None None] MFA Rules not processed for user `0a813031f734439f91679e893f7b265f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:09.114460 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-466df4de-89c7-4431-8ed8-fe98cc134d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:09.177580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1088/2169] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:09.188520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:09.190170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:09.210828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:09.247175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46702a7a-5579-42b0-9e5c-223f92bf6964 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:09.250783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3044226-d42a-46d1-bdaa-065ac4ec4567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:09.251375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3044226-d42a-46d1-bdaa-065ac4ec4567 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.251651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3044226-d42a-46d1-bdaa-065ac4ec4567 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.251927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3044226-d42a-46d1-bdaa-065ac4ec4567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.260332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3044226-d42a-46d1-bdaa-065ac4ec4567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:09.261740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3044226-d42a-46d1-bdaa-065ac4ec4567 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:09.271431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1082/2170] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:09.300738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466df4de-89c7-4431-8ed8-fe98cc134d99 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:09.320787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:09.364804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466df4de-89c7-4431-8ed8-fe98cc134d99 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:09.369405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:09.370021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.370297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.370568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] PATH_INFO: `/v3/users/0a813031f734439f91679e893f7b265f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.371677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a813031f734439f91679e893f7b265f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:09.373622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:09.418250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a813031f734439f91679e893f7b265f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.425081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a813031f734439f91679e893f7b265f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.432102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a813031f734439f91679e893f7b265f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.432743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a813031f734439f91679e893f7b265f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.444720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a813031f734439f91679e893f7b265f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.451585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1089/2171] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:00:09 2022] DELETE /identity/v3/users/0a813031f734439f91679e893f7b265f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:09.455842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.456334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.456706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.477516 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:09.479301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:09.539655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1083/2172] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:09.549516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:09.550886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:09.615126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f166-4507-434b-8901-9f9fcf6f6488 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:09.618723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bc533-0794-4636-8e19-7da55cb3a061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:09.619321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc533-0794-4636-8e19-7da55cb3a061 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.619597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc533-0794-4636-8e19-7da55cb3a061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.619863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc533-0794-4636-8e19-7da55cb3a061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.628754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bc533-0794-4636-8e19-7da55cb3a061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:09.630424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bc533-0794-4636-8e19-7da55cb3a061 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:09.640153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1090/2173] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:09.900198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:09.902203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:09.915139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7f5a-1e49-4248-ae40-960d37a63cd8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:09.920346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:09.921001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:09.921273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:09.921546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] PATH_INFO: `/v3/projects/12181383c83046ef96403279e1d25147` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:09.922695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12181383c83046ef96403279e1d25147)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:09.924657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:09.971891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12181383c83046ef96403279e1d25147', 'request_id': 'req-50723158-d299-4c23-9516-949d34b315a8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.979064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12181383c83046ef96403279e1d25147', 'request_id': 'req-50723158-d299-4c23-9516-949d34b315a8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.984441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12181383c83046ef96403279e1d25147', 'request_id': 'req-50723158-d299-4c23-9516-949d34b315a8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.995484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] The token cache is being invalidate because project 12181383c83046ef96403279e1d25147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:00:09.995768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.996413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-50723158-d299-4c23-9516-949d34b315a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:09.997880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1084/2174] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:00:09 2022] DELETE /identity/v3/projects/12181383c83046ef96403279e1d25147 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:10.002859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300efb4c-329f-4cbe-96ce-d1f4aad56ec7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.003358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300efb4c-329f-4cbe-96ce-d1f4aad56ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.003358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300efb4c-329f-4cbe-96ce-d1f4aad56ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.022998 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-300efb4c-329f-4cbe-96ce-d1f4aad56ec7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:10.024175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-300efb4c-329f-4cbe-96ce-d1f4aad56ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:10.054754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:10.081707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1091/2175] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:10.089763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff4fca-4526-4ba0-a66c-0f0430281df2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.089763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff4fca-4526-4ba0-a66c-0f0430281df2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.090212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff4fca-4526-4ba0-a66c-0f0430281df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.111286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2cff4fca-4526-4ba0-a66c-0f0430281df2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:10.111878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2cff4fca-4526-4ba0-a66c-0f0430281df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:10.120447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1085/2176] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:10.124904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300efb4c-329f-4cbe-96ce-d1f4aad56ec7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:10.135545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300efb4c-329f-4cbe-96ce-d1f4aad56ec7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:10.140382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:10.141082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.141425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.141787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.142503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:00:10.142868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:10.144563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:10.156284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1092/2177] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:00:10.160996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff4fca-4526-4ba0-a66c-0f0430281df2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:10.171362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff4fca-4526-4ba0-a66c-0f0430281df2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:10.178124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:10.178800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.179152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.179514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.180224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:10.182765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:10.212060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1086/2178] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:00:10 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:00:10.216491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:10.226318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59faec-cd09-43b3-90e9-2d896d4a1655 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:10.280762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:10.281286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.281559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.281857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.282513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:10.284442 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:10.306956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1093/2179] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:00:10 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:00:10.311211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:10.321239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe34f0-61e5-441b-9b2a-b8884b7a101c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:10.326140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:10.326744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.327078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.327359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.328196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:10.329839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:10.340951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1087/2180] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:00:10.344713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:10.355123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1b45-a65d-4400-a7d2-c384deff823d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:10.359836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de6f79-f8af-4c0b-9732-06360d9801af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:10.360463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de6f79-f8af-4c0b-9732-06360d9801af admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.360739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de6f79-f8af-4c0b-9732-06360d9801af admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.361018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de6f79-f8af-4c0b-9732-06360d9801af admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.361653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de6f79-f8af-4c0b-9732-06360d9801af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:10.363230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de6f79-f8af-4c0b-9732-06360d9801af admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:10.374379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1094/2181] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:00:10.381730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:10.391859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0fbb4-7a76-4ee6-8bc6-411ac0d6a91e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:10.396624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:10.397265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.397550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.397828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] PATH_INFO: `/v3/projects/518ae9ee2701464bb18625b29991a9c3/users/9eb2ee88c0c64cb6a6b6251eb48c09e9/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.400034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:10.415975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518ae9ee2701464bb18625b29991a9c3, user_id=9eb2ee88c0c64cb6a6b6251eb48c09e9, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:10.418737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:10.436080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1088/2182] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:00:10 2022] PUT /identity/v3/projects/518ae9ee2701464bb18625b29991a9c3/users/9eb2ee88c0c64cb6a6b6251eb48c09e9/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:10.446073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.446360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.446632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.473134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] MFA Rules not processed for user `9eb2ee88c0c64cb6a6b6251eb48c09e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:10.474329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:10.536041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1095/2183] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:10.547040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:10.548714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:10.603428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1fdb-e57f-463b-9edf-7cffb16424c4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:10.607031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:10.607620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:10.607897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:10.608167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:10.616522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:10.617932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:10.628943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1089/2184] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:20.059005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:20.060418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:20.098202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:20.116368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb14b5e-1836-4b35-a24f-dae54f5cee68 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:20.120564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:20.121157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:20.121437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:20.121707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:20.123384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:20.190838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:20.192322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:20.202897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1096/2185] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:00:20.623894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:22.239804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:22.241343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:22.254475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27368371-e9cf-4da0-a42e-4cb6ab685330 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:22.258364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:22.258997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:22.259267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:22.259533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:22.261266 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:22.275525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:22.276977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:22.290998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1090/2186] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:29.299063 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:29.300497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:29.314577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87d5dd-0921-4612-a91a-fd5af2c234b8 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:29.319945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:29.320571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:29.320849 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:29.321148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:29.322784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:29.378706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:29.380579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:29.389401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1097/2187] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:32.284359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:35.672564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:35.674119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:35.689118 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27588bdf-6f94-43d9-9c58-e92bb8b6a584 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:35.694863 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:35.695464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:35.695738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:35.696647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:35.698760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:35.767871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:35.769619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:35.780858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1091/2188] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:00:37.964325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:37.975185 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d29a5d-206b-4571-87d1-d0f78bcc4eea service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:37.981293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4e0d8-b0e0-4514-a3fc-c8eb89113664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:37.981929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e0d8-b0e0-4514-a3fc-c8eb89113664 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:37.982210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e0d8-b0e0-4514-a3fc-c8eb89113664 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:37.982474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e0d8-b0e0-4514-a3fc-c8eb89113664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:37.984195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-30e4e0d8-b0e0-4514-a3fc-c8eb89113664 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:38.046080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4e0d8-b0e0-4514-a3fc-c8eb89113664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:38.047629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4e0d8-b0e0-4514-a3fc-c8eb89113664 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:38.056570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1098/2189] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:40.489815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:40.558202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701b6df-8a94-4595-93e1-e91d41c2e335 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:40.563170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:40.563793 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:40.564070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:40.564393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] PATH_INFO: `/v3/users/9eb2ee88c0c64cb6a6b6251eb48c09e9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:40.565518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb2ee88c0c64cb6a6b6251eb48c09e9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:40.567552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:40.620715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb2ee88c0c64cb6a6b6251eb48c09e9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:40.632757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb2ee88c0c64cb6a6b6251eb48c09e9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:40.642998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb2ee88c0c64cb6a6b6251eb48c09e9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:40.643735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb2ee88c0c64cb6a6b6251eb48c09e9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:40.644362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb2ee88c0c64cb6a6b6251eb48c09e9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:40.660671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1092/2190] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:00:40 2022] DELETE /identity/v3/users/9eb2ee88c0c64cb6a6b6251eb48c09e9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:40.665193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:40.665704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:40.665704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:40.685504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:40.686680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:40.694685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:40.746408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1099/2191] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:00:40.756270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:40.757695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:40.778324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:40.815819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaacb9c-0239-473f-879d-eba81d74eb24 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:40.819442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c8cf0-7925-4f06-91e4-616a3ce5f2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:40.820036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c8cf0-7925-4f06-91e4-616a3ce5f2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:40.820342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c8cf0-7925-4f06-91e4-616a3ce5f2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:40.820621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c8cf0-7925-4f06-91e4-616a3ce5f2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:40.829881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c8cf0-7925-4f06-91e4-616a3ce5f2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:40.831286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c8cf0-7925-4f06-91e4-616a3ce5f2c9 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:40.840624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1093/2192] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:41.095666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.097236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:41.109282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0511f3e-900f-4844-9fa5-da20dc6d086d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.114519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.115388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.115594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.119532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] PATH_INFO: `/v3/projects/518ae9ee2701464bb18625b29991a9c3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.119532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518ae9ee2701464bb18625b29991a9c3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.119532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.168070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518ae9ee2701464bb18625b29991a9c3', 'request_id': 'req-2dc14cde-eb07-4bdc-af58-0dc16f539b43'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:41.174043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518ae9ee2701464bb18625b29991a9c3', 'request_id': 'req-2dc14cde-eb07-4bdc-af58-0dc16f539b43'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:41.179425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518ae9ee2701464bb18625b29991a9c3', 'request_id': 'req-2dc14cde-eb07-4bdc-af58-0dc16f539b43'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:41.191888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] The token cache is being invalidate because project 518ae9ee2701464bb18625b29991a9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:00:41.192170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:41.192825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2dc14cde-eb07-4bdc-af58-0dc16f539b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:41.194289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1100/2193] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:00:41 2022] DELETE /identity/v3/projects/518ae9ee2701464bb18625b29991a9c3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:41.199175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a9db-558e-4a55-8468-d2345d9d8aef None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.199463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a9db-558e-4a55-8468-d2345d9d8aef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.199738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a9db-558e-4a55-8468-d2345d9d8aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.218821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c2f8a9db-558e-4a55-8468-d2345d9d8aef None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:41.220005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c2f8a9db-558e-4a55-8468-d2345d9d8aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:41.278884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1094/2194] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:41.286640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d805e-d44c-4f82-a974-c99fefea079e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.286923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d805e-d44c-4f82-a974-c99fefea079e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.287192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d805e-d44c-4f82-a974-c99fefea079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.310433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a51d805e-d44c-4f82-a974-c99fefea079e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:41.312291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a51d805e-d44c-4f82-a974-c99fefea079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:41.320444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1101/2195] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:41.324564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a9db-558e-4a55-8468-d2345d9d8aef None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.337547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a9db-558e-4a55-8468-d2345d9d8aef None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.342169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.342780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.343061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.343335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.343986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:00:41.344316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.345900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.362274 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1095/2196] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:00:41.363317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d805e-d44c-4f82-a974-c99fefea079e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.373308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d805e-d44c-4f82-a974-c99fefea079e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.380922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.382977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.383385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.383666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.384398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.387049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.418980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1102/2197] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:00:41 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:00:41.422945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.436207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e45b-717e-421e-a5bf-9b865ef3dcff admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.500913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.501608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.501789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.502064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.502713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.504902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.527420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1096/2198] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:00:41 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:00:41.531470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.542332 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd6ed1-ddca-4dc7-9bb5-6c6be3c4bacb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.546948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.547550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.547829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.548103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.548779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.550418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.560601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1103/2199] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:00:41.564655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.574920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15088e05-3bca-42e8-b44d-1caf05d1b277 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.579423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2df97-4cf9-4322-8126-681558fe8a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.580005 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2df97-4cf9-4322-8126-681558fe8a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.580307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2df97-4cf9-4322-8126-681558fe8a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.580580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2df97-4cf9-4322-8126-681558fe8a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.581207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2df97-4cf9-4322-8126-681558fe8a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.582804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2df97-4cf9-4322-8126-681558fe8a82 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.593499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1097/2200] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:00:41.597248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.606905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736037e9-7f63-4b3a-9d40-92863c7e6266 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.613374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.614048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.614323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.614599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] PATH_INFO: `/v3/projects/7f1f11e793c34b518e4de64594c7b649/users/babdb46202ab41e9b08caf3faceb3bcf/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.630088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1f11e793c34b518e4de64594c7b649, user_id=babdb46202ab41e9b08caf3faceb3bcf, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.633018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.647275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1104/2201] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:00:41 2022] PUT /identity/v3/projects/7f1f11e793c34b518e4de64594c7b649/users/babdb46202ab41e9b08caf3faceb3bcf/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:41.657399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.657688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.657965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.685974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] MFA Rules not processed for user `babdb46202ab41e9b08caf3faceb3bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:41.687112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:41.700369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:41.765815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1098/2202] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:41 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:41.777847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:41.779602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:41.815139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:41.840281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad868a0-2bbb-4a89-ab0e-636554edd823 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:41.843790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0be475-6a07-4e6e-89b2-78e36510b744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:41.844437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0be475-6a07-4e6e-89b2-78e36510b744 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:41.844715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0be475-6a07-4e6e-89b2-78e36510b744 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:41.844993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0be475-6a07-4e6e-89b2-78e36510b744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:41.854050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0be475-6a07-4e6e-89b2-78e36510b744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:41.855535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0be475-6a07-4e6e-89b2-78e36510b744 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:41.864833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1105/2203] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:42.526479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:42.530267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:42.614286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af13fc-190e-4ac7-a81b-a4480680d6d9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:42.618010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264193d3-0aac-4642-8677-eec1177374e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:42.619237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264193d3-0aac-4642-8677-eec1177374e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:42.619534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264193d3-0aac-4642-8677-eec1177374e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:42.619810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264193d3-0aac-4642-8677-eec1177374e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:42.621757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-264193d3-0aac-4642-8677-eec1177374e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:42.636325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264193d3-0aac-4642-8677-eec1177374e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:42.638199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264193d3-0aac-4642-8677-eec1177374e1 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:42.648014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1099/2204] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:42.796367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ad798-eb50-4569-bd02-a70c47096f1d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:42.796858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ad798-eb50-4569-bd02-a70c47096f1d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:42.797060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ad798-eb50-4569-bd02-a70c47096f1d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:42.798593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1106/2205] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 02:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 02:00:42.805342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa015-852e-49cd-babf-55f65b9ccd7f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:42.805857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa015-852e-49cd-babf-55f65b9ccd7f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:42.806366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa015-852e-49cd-babf-55f65b9ccd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:42.848542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-150aa015-852e-49cd-babf-55f65b9ccd7f None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:42.849918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-150aa015-852e-49cd-babf-55f65b9ccd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:42.951525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1100/2206] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 02:00:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:00:42.968376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ad798-eb50-4569-bd02-a70c47096f1d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:42.971277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0a5ad798-eb50-4569-bd02-a70c47096f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:43.055866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ad798-eb50-4569-bd02-a70c47096f1d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:43.059433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489680f2-08f8-46e3-969a-7e4ba44909cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:43.060143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489680f2-08f8-46e3-969a-7e4ba44909cb service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.060450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489680f2-08f8-46e3-969a-7e4ba44909cb service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.060720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489680f2-08f8-46e3-969a-7e4ba44909cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.073118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489680f2-08f8-46e3-969a-7e4ba44909cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:43.074635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489680f2-08f8-46e3-969a-7e4ba44909cb service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:43.086182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1107/2207] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:43.162643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fedb65-f26c-4543-aa8a-d7f20869d440 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.162832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fedb65-f26c-4543-aa8a-d7f20869d440 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.163427 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fedb65-f26c-4543-aa8a-d7f20869d440 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.164792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1101/2208] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 02:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 02:00:43.170017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c4f0-16e6-46f6-b419-2feeb588b17b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.170478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c4f0-16e6-46f6-b419-2feeb588b17b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.170942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9c4f0-16e6-46f6-b419-2feeb588b17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.191759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-92c9c4f0-16e6-46f6-b419-2feeb588b17b None None] MFA Rules not processed for user `e1c3b7f7c83f48aca27fbd46202f6506`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:43.193028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-92c9c4f0-16e6-46f6-b419-2feeb588b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:43.200206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1108/2209] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 02:00:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:00:43.212108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fedb65-f26c-4543-aa8a-d7f20869d440 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:43.213811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-50fedb65-f26c-4543-aa8a-d7f20869d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:43.226969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fedb65-f26c-4543-aa8a-d7f20869d440 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:43.230873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb3fb4-9f23-40ad-930f-a12ae6d0514b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:43.231470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb3fb4-9f23-40ad-930f-a12ae6d0514b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.231752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb3fb4-9f23-40ad-930f-a12ae6d0514b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.232031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb3fb4-9f23-40ad-930f-a12ae6d0514b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.239909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb3fb4-9f23-40ad-930f-a12ae6d0514b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:43.241403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb3fb4-9f23-40ad-930f-a12ae6d0514b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:43.250386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1102/2210] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:43.334456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac4c0f-609c-4887-b15e-46f7e9423cd2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.334884 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac4c0f-609c-4887-b15e-46f7e9423cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.335364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac4c0f-609c-4887-b15e-46f7e9423cd2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.337557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1109/2211] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 02:00:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 02:00:43.343867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.344178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.344498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.365292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:43.366510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:43.376960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1103/2212] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 02:00:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:43.388692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac4c0f-609c-4887-b15e-46f7e9423cd2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:43.400193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac4c0f-609c-4887-b15e-46f7e9423cd2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:43.404517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:43.405114 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.405396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.405672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.416287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:43.417703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:43.428412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1110/2213] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:43.495095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:43.496519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:43.535885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:43.574418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26307c1d-d963-4f7c-a2a5-7e5d477be46b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:43.580404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21272fe-daa3-4e4e-a3cf-78248dc48137 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:43.581015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21272fe-daa3-4e4e-a3cf-78248dc48137 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:43.581293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21272fe-daa3-4e4e-a3cf-78248dc48137 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:43.581567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21272fe-daa3-4e4e-a3cf-78248dc48137 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:43.583318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c21272fe-daa3-4e4e-a3cf-78248dc48137 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:43.596165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21272fe-daa3-4e4e-a3cf-78248dc48137 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:43.597798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21272fe-daa3-4e4e-a3cf-78248dc48137 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:43.608529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1104/2214] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:53.423609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=46 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:53.603952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=18 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:55.228830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:55.293423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e63bc-8e18-4737-991c-e4067f9a5285 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:55.298443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:55.298871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:55.299148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:55.299425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] PATH_INFO: `/v3/users/babdb46202ab41e9b08caf3faceb3bcf` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:55.300599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=babdb46202ab41e9b08caf3faceb3bcf)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:55.302536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:55.349321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'babdb46202ab41e9b08caf3faceb3bcf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:55.356292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'babdb46202ab41e9b08caf3faceb3bcf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:55.364318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'babdb46202ab41e9b08caf3faceb3bcf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:55.365004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'babdb46202ab41e9b08caf3faceb3bcf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:55.376729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'babdb46202ab41e9b08caf3faceb3bcf'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:55.382790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1111/2215] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:00:55 2022] DELETE /identity/v3/users/babdb46202ab41e9b08caf3faceb3bcf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:55.387754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:55.388293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:55.388405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:55.407644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:55.408865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:55.467981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1105/2216] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:55.477710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:55.479257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:55.546840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eeb462-e23f-4003-b409-af08f445f848 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:55.550790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15859da-0bd8-4d9e-a8eb-fde7929d5bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:55.551403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15859da-0bd8-4d9e-a8eb-fde7929d5bec service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:55.551687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15859da-0bd8-4d9e-a8eb-fde7929d5bec service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:55.551964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15859da-0bd8-4d9e-a8eb-fde7929d5bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:55.561386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15859da-0bd8-4d9e-a8eb-fde7929d5bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:55.562858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15859da-0bd8-4d9e-a8eb-fde7929d5bec service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:55.573115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1112/2217] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:00:55.943100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:55.944586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:55.968066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14224434-bfdf-414e-a0f3-526e26160554 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:55.981071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:55.981648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:55.981936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:55.982205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] PATH_INFO: `/v3/projects/7f1f11e793c34b518e4de64594c7b649` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:55.983381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1f11e793c34b518e4de64594c7b649)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:55.985335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.081433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1f11e793c34b518e4de64594c7b649', 'request_id': 'req-30a0627a-db67-4631-b754-6313165c5f53'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:56.089622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1f11e793c34b518e4de64594c7b649', 'request_id': 'req-30a0627a-db67-4631-b754-6313165c5f53'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:56.098127 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1f11e793c34b518e4de64594c7b649', 'request_id': 'req-30a0627a-db67-4631-b754-6313165c5f53'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:56.112414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] The token cache is being invalidate because project 7f1f11e793c34b518e4de64594c7b649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:00:56.112705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:56.113365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30a0627a-db67-4631-b754-6313165c5f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:00:56.115055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1106/2218] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:00:55 2022] DELETE /identity/v3/projects/7f1f11e793c34b518e4de64594c7b649 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:56.120564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b05af8-ee8c-4968-8c5d-364c79b73453 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.120986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b05af8-ee8c-4968-8c5d-364c79b73453 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.121365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b05af8-ee8c-4968-8c5d-364c79b73453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.152661 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-96b05af8-ee8c-4968-8c5d-364c79b73453 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:56.154041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-96b05af8-ee8c-4968-8c5d-364c79b73453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:56.181595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:56.237952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1113/2219] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:56.247221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04264f-24b0-4a0f-b000-a483e3cb373f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.247523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04264f-24b0-4a0f-b000-a483e3cb373f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.247786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04264f-24b0-4a0f-b000-a483e3cb373f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.270307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cf04264f-24b0-4a0f-b000-a483e3cb373f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:56.271544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cf04264f-24b0-4a0f-b000-a483e3cb373f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:56.284099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1107/2220] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:56.285358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b05af8-ee8c-4968-8c5d-364c79b73453 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:56.296477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b05af8-ee8c-4968-8c5d-364c79b73453 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:56.301778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:56.302197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.302480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.302751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.303392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:00:56.304008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:56.306400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.317881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1114/2221] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:00:56.322215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04264f-24b0-4a0f-b000-a483e3cb373f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:56.331776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04264f-24b0-4a0f-b000-a483e3cb373f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:56.336532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:56.337165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.337444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.337717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.338384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:56.340318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.371935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1108/2222] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:00:56 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:00:56.376200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:56.386718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65106f25-be52-4373-9877-fb1850cf402e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:56.444990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:56.445625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.445904 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.446219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.446852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:56.448817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.471087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1115/2223] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:00:56 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:00:56.475303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:56.486230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3cbf9-64b3-4cb0-86e0-a2097f3841dc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:56.491223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:56.491819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.492129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.492456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.493096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:56.494722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.504561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1109/2224] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:00:56.508733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:56.518815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c8d2b-0951-44ce-ac0a-44dfb53a2b93 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:56.523540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1502ba-55af-4f9d-a73f-d6cf875e3d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:56.524720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1502ba-55af-4f9d-a73f-d6cf875e3d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.524984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1502ba-55af-4f9d-a73f-d6cf875e3d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.525252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1502ba-55af-4f9d-a73f-d6cf875e3d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.525892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1502ba-55af-4f9d-a73f-d6cf875e3d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:56.527545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1502ba-55af-4f9d-a73f-d6cf875e3d99 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.543257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1116/2225] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:00:56.544583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:56.554930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a23df4-2858-4162-ac4b-b34ad7345003 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:56.559916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:56.560610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.560886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.561157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] PATH_INFO: `/v3/projects/74610e8b7b4d4fafa0e49ba73b94aa48/users/bd08361520db465bb9069326d03d0307/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.577030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74610e8b7b4d4fafa0e49ba73b94aa48, user_id=bd08361520db465bb9069326d03d0307, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:56.579593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.592918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1110/2226] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:00:56 2022] PUT /identity/v3/projects/74610e8b7b4d4fafa0e49ba73b94aa48/users/bd08361520db465bb9069326d03d0307/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:00:56.598004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd63d-7ee2-4312-8bc6-c4dedea825dd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.598290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd63d-7ee2-4312-8bc6-c4dedea825dd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.598561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fd63d-7ee2-4312-8bc6-c4dedea825dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.618930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ce3fd63d-7ee2-4312-8bc6-c4dedea825dd None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:00:56.620129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ce3fd63d-7ee2-4312-8bc6-c4dedea825dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:56.679503 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1117/2227] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:00:56.690364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:00:56.692055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:00:56.714538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:00:56.752332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066156d-91d0-494a-aef5-fbc173afc372 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:00:56.756193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:00:56.756818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:00:56.757268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:00:56.757579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:00:56.767575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:00:56.769035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:00:56.778601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1111/2228] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:01:00.433625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fd63d-7ee2-4312-8bc6-c4dedea825dd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:00.444558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:00.454050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c146-e55e-447a-9241-5234dc253d67 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:00.454838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fd63d-7ee2-4312-8bc6-c4dedea825dd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:00.467527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe707bdf-15ca-473b-9b34-e94ca89254b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:00.470725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe707bdf-15ca-473b-9b34-e94ca89254b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:00.471572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f15eff-3931-45e8-ac5b-6a41c37500b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:00.472411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15eff-3931-45e8-ac5b-6a41c37500b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:00.472822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe707bdf-15ca-473b-9b34-e94ca89254b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:00.473154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe707bdf-15ca-473b-9b34-e94ca89254b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:00.474446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15eff-3931-45e8-ac5b-6a41c37500b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:00.474771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f15eff-3931-45e8-ac5b-6a41c37500b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:00.480374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fe707bdf-15ca-473b-9b34-e94ca89254b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:00.482341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-60f15eff-3931-45e8-ac5b-6a41c37500b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:00.504270 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f15eff-3931-45e8-ac5b-6a41c37500b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:00.511815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f15eff-3931-45e8-ac5b-6a41c37500b8 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:00.516643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe707bdf-15ca-473b-9b34-e94ca89254b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:00.518756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe707bdf-15ca-473b-9b34-e94ca89254b4 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:00.539165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1112/2229] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:01:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:01:00.540036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1118/2230] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:01:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:01:01.195304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015ef09-ae15-41ef-a354-48610493aece None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:01.195304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015ef09-ae15-41ef-a354-48610493aece None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:01.195705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015ef09-ae15-41ef-a354-48610493aece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:01.232074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8015ef09-ae15-41ef-a354-48610493aece None None] MFA Rules not processed for user `0646d81951fa4353bdfb59048b498652`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:01:01.234086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8015ef09-ae15-41ef-a354-48610493aece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:01.339600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1113/2231] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:01:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:01:01.397536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6bed-f634-442c-8359-58583c93f191 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:01.397536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6bed-f634-442c-8359-58583c93f191 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:01.397789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6bed-f634-442c-8359-58583c93f191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:01.456647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0c7a6bed-f634-442c-8359-58583c93f191 None None] MFA Rules not processed for user `bd08361520db465bb9069326d03d0307`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:01:01.459073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0c7a6bed-f634-442c-8359-58583c93f191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:01.465025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:01:01.590961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1119/2232] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:01:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:01:01.591265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015ef09-ae15-41ef-a354-48610493aece None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:01.605542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015ef09-ae15-41ef-a354-48610493aece None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:01.610742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:01.611896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:01.612572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:01.613859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:01.614771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:01.617032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:01.664659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6bed-f634-442c-8359-58583c93f191 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:01.672177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1114/2233] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:01:01 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:01:01.681451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6bed-f634-442c-8359-58583c93f191 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:01.774714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:01.775341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:01.775618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:01.775893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:01.776575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:01.778600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:01.786419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:01.788506 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:01.811786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1120/2234] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:01:01 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:01:01.813190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:01.832367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e871b-0b92-45bd-9287-600433bd0b87 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:01.838474 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:01.839342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:01.839945 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:01.840590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:01.841476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:01.844041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:01.862006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1121/2235] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:01:01.862230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469a5dc-12e3-47e5-9b75-088910b6267a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:01.863614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:01.865518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:01.866395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:01.866896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:01.867894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:01.869896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:01.877006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b7d6e-811e-4902-bdd7-8bf0aa468418 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:01.884397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:01.885174 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:01.885458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:01.885802 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] PATH_INFO: `/v3/projects/d75ab1367ec047f996b8063a07f68220/users/debecc8a5c6b4de08aa0d5c8dea91362/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:01.897293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:01.899805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:01.909523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1115/2236] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:01:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:01:01.920292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75ab1367ec047f996b8063a07f68220, user_id=debecc8a5c6b4de08aa0d5c8dea91362, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:01.923181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:01.940102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1122/2237] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:01:01 2022] PUT /identity/v3/projects/d75ab1367ec047f996b8063a07f68220/users/debecc8a5c6b4de08aa0d5c8dea91362/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:01:01.943919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:01.957689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e43b02-da2c-40ac-a420-23a8144cf08b service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:02.023227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:01:02.058789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051a62c-4587-411f-9131-d8bda8928c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:02.059464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051a62c-4587-411f-9131-d8bda8928c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:02.059743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051a62c-4587-411f-9131-d8bda8928c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:02.060018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051a62c-4587-411f-9131-d8bda8928c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:02.060750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051a62c-4587-411f-9131-d8bda8928c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:02.062404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051a62c-4587-411f-9131-d8bda8928c5f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:02.073358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1116/2238] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:01:02.078116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:02.089713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451eb5c-4deb-4fbf-b676-50af28d7685f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:02.095094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:02.095971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:02.096350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:02.096688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] PATH_INFO: `/v3/projects/d75ab1367ec047f996b8063a07f68220/users/debecc8a5c6b4de08aa0d5c8dea91362/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:02.099035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75ab1367ec047f996b8063a07f68220, user_id=debecc8a5c6b4de08aa0d5c8dea91362, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:02.102368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:02.115234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1123/2239] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:01:02 2022] PUT /identity/v3/projects/d75ab1367ec047f996b8063a07f68220/users/debecc8a5c6b4de08aa0d5c8dea91362/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:01:02.255124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad837f7-96b1-4235-a399-68b9a84dc988 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:02.255733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad837f7-96b1-4235-a399-68b9a84dc988 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:02.256112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad837f7-96b1-4235-a399-68b9a84dc988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:02.282532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7ad837f7-96b1-4235-a399-68b9a84dc988 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:01:02.284909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ad837f7-96b1-4235-a399-68b9a84dc988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:02.382540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1117/2240] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 02:01:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:01:02.455697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:02.457772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:02.541699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:01:02.582374 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f6671-b843-411c-a4b3-cc4ce531689e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:02.590648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:02.591439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:02.591808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:02.592367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:02.605030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:02.607107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:02.619200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1124/2241] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:01:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:01:08.713492 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f4fa9-359d-4283-9bf7-65100d4742d9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:08.714036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f4fa9-359d-4283-9bf7-65100d4742d9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:08.714036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f4fa9-359d-4283-9bf7-65100d4742d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:08.749937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-046f4fa9-359d-4283-9bf7-65100d4742d9 None None] MFA Rules not processed for user `debecc8a5c6b4de08aa0d5c8dea91362`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:01:08.751331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-046f4fa9-359d-4283-9bf7-65100d4742d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:08.843013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1118/2242] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:01:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:01:08.856830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:08.858321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:08.965292 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee0e74-bf16-454f-8f88-b0fddd94a907 service swift] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:08.977026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:08.980057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:08.980528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:08.981289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:09.064224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:09.066058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:09.077413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1125/2243] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:01:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:01:18.840278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=72 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:01:19.072248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=67 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:01:26.325111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f4fa9-359d-4283-9bf7-65100d4742d9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:26.340131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f4fa9-359d-4283-9bf7-65100d4742d9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:26.347012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f9f15-4c2d-48ab-9091-a4ab3f2cbee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:26.347900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f9f15-4c2d-48ab-9091-a4ab3f2cbee1 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:26.348396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f9f15-4c2d-48ab-9091-a4ab3f2cbee1 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:26.348714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f9f15-4c2d-48ab-9091-a4ab3f2cbee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:26.350764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ad3f9f15-4c2d-48ab-9091-a4ab3f2cbee1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:26.363853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f9f15-4c2d-48ab-9091-a4ab3f2cbee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:26.365457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f9f15-4c2d-48ab-9091-a4ab3f2cbee1 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:26.376534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1119/2244] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:01:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:01:35.935613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:35.947833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12984a-a1a8-4b19-ab83-f0a08c63e320 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:35.953584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc63ba-1e98-4ebc-add6-f9cff6b5a1a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:35.954168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc63ba-1e98-4ebc-add6-f9cff6b5a1a8 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:35.954443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc63ba-1e98-4ebc-add6-f9cff6b5a1a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:35.954713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc63ba-1e98-4ebc-add6-f9cff6b5a1a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:35.956445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-61cc63ba-1e98-4ebc-add6-f9cff6b5a1a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:35.969312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc63ba-1e98-4ebc-add6-f9cff6b5a1a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:35.970891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc63ba-1e98-4ebc-add6-f9cff6b5a1a8 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:35.981300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1126/2245] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:01:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:01:36.372406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:01:38.859019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c07b-fade-4189-9194-1236cc2a0428 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:38.859612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c07b-fade-4189-9194-1236cc2a0428 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:38.859698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c07b-fade-4189-9194-1236cc2a0428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:38.878922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-49d9c07b-fade-4189-9194-1236cc2a0428 None None] MFA Rules not processed for user `bd08361520db465bb9069326d03d0307`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:01:38.880131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-49d9c07b-fade-4189-9194-1236cc2a0428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:38.887428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1120/2246] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:01:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:01:44.358100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb56d88-db5b-488f-aa1b-4d41c7e1c85d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:44.358954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb56d88-db5b-488f-aa1b-4d41c7e1c85d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:44.359376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb56d88-db5b-488f-aa1b-4d41c7e1c85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:44.380966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7eb56d88-db5b-488f-aa1b-4d41c7e1c85d None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:01:44.382466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7eb56d88-db5b-488f-aa1b-4d41c7e1c85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:01:44.397278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1127/2247] 10.222.0.50 () {58 vars in 967 bytes} [Tue Aug 30 02:01:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:01:44.405796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9c07b-fade-4189-9194-1236cc2a0428 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:01:44.418585 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9c07b-fade-4189-9194-1236cc2a0428 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:01:44.424120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:01:44.424909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:01:44.425296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:01:44.425666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:01:44.436315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:01:44.437928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:01:44.447270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1121/2248] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:01:54.374421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=9 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:01:54.443578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=9 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:15.607688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb56d88-db5b-488f-aa1b-4d41c7e1c85d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:15.620314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb56d88-db5b-488f-aa1b-4d41c7e1c85d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:15.625714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:15.626316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:15.626592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:15.626865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:15.629237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:15.688118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:15.689780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:15.699719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1128/2249] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:02:18.243011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:18.299725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a92a8-740a-4b07-8a18-b87244340971 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:18.303322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb617c06-ba92-4d41-b2a2-99b86405f90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:18.303961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617c06-ba92-4d41-b2a2-99b86405f90a service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:18.304255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617c06-ba92-4d41-b2a2-99b86405f90a service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:18.304542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb617c06-ba92-4d41-b2a2-99b86405f90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:18.313938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb617c06-ba92-4d41-b2a2-99b86405f90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:18.315371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb617c06-ba92-4d41-b2a2-99b86405f90a service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:18.325259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1122/2250] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:02:25.695081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=28 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:26.923180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:26.934432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8da93-358c-4e36-af03-2d7f80f7045d service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:26.940087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bcf8b-0c14-4824-a712-76158f11c7a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:26.940721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcf8b-0c14-4824-a712-76158f11c7a3 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:26.940995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcf8b-0c14-4824-a712-76158f11c7a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:26.941262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcf8b-0c14-4824-a712-76158f11c7a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:26.943072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8c0bcf8b-0c14-4824-a712-76158f11c7a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:27.001665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bcf8b-0c14-4824-a712-76158f11c7a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:27.003253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bcf8b-0c14-4824-a712-76158f11c7a3 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:27.013011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1129/2251] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:02:28.321152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=26 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:37.008890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=28 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:43.500821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec48b-2892-468a-87e9-7f95d28536bc None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:43.501354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec48b-2892-468a-87e9-7f95d28536bc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:43.501354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec48b-2892-468a-87e9-7f95d28536bc None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:43.502714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1123/2252] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 02:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 02:02:43.512126 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25d7e5-1fd8-4bf2-87d6-a30ae0a706bc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:43.512708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25d7e5-1fd8-4bf2-87d6-a30ae0a706bc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:43.512809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25d7e5-1fd8-4bf2-87d6-a30ae0a706bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:43.535064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-eb25d7e5-1fd8-4bf2-87d6-a30ae0a706bc None None] MFA Rules not processed for user `e1c3b7f7c83f48aca27fbd46202f6506`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:43.536309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eb25d7e5-1fd8-4bf2-87d6-a30ae0a706bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:43.542801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1130/2253] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 02:02:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:43.554836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec48b-2892-468a-87e9-7f95d28536bc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:43.565470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec48b-2892-468a-87e9-7f95d28536bc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:43.569582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fd918-3299-4531-bab1-e4d9cf7ed4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:43.570201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fd918-3299-4531-bab1-e4d9cf7ed4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:43.570476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fd918-3299-4531-bab1-e4d9cf7ed4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:43.570741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fd918-3299-4531-bab1-e4d9cf7ed4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:43.577860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fd918-3299-4531-bab1-e4d9cf7ed4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:43.579260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fd918-3299-4531-bab1-e4d9cf7ed4f6 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:43.589305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1124/2254] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:02:43.670831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8db15-a14b-40ff-8695-fb9f38a2ca30 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:43.671117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8db15-a14b-40ff-8695-fb9f38a2ca30 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:43.671392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8db15-a14b-40ff-8695-fb9f38a2ca30 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:43.672790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1131/2255] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 02:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 02:02:43.677461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd9266-56d7-45a2-ad79-0c6c710d5097 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:43.677745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd9266-56d7-45a2-ad79-0c6c710d5097 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:43.678020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd9266-56d7-45a2-ad79-0c6c710d5097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:43.698423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5ccd9266-56d7-45a2-ad79-0c6c710d5097 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:43.699601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5ccd9266-56d7-45a2-ad79-0c6c710d5097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:43.708986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1125/2256] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 02:02:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:43.719326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8db15-a14b-40ff-8695-fb9f38a2ca30 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:43.729984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8db15-a14b-40ff-8695-fb9f38a2ca30 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:43.733518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954033b-ace5-4e9a-9d80-3c30f7a61e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:43.734096 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954033b-ace5-4e9a-9d80-3c30f7a61e85 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:43.734366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954033b-ace5-4e9a-9d80-3c30f7a61e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:43.734631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954033b-ace5-4e9a-9d80-3c30f7a61e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:43.745426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954033b-ace5-4e9a-9d80-3c30f7a61e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:43.746850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954033b-ace5-4e9a-9d80-3c30f7a61e85 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:43.758157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1132/2257] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:02:48.626771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330b22e-686e-4b4b-b807-211226725bc8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:48.627354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330b22e-686e-4b4b-b807-211226725bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:48.627354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330b22e-686e-4b4b-b807-211226725bc8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:48.628809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1126/2258] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 02:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 02:02:48.656653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:48.657149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:48.657372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:48.677469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] MFA Rules not processed for user `588e9ed6789e46d6a6693cb54e6fdce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:48.678657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:48.690715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330b22e-686e-4b4b-b807-211226725bc8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:48.692438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e330b22e-686e-4b4b-b807-211226725bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:48.753531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1133/2259] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 02:02:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:48.765201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:48.766616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:48.766750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330b22e-686e-4b4b-b807-211226725bc8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:48.771149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:48.771773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:48.772095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:48.772410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] PATH_INFO: `/v3/users/0646d81951fa4353bdfb59048b498652` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:48.774198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0646d81951fa4353bdfb59048b498652)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:48.776220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:48.779965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cb749-9879-44f6-a13e-3a7edfbc5cf9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:48.785513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:48.786135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:48.786419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:48.786700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:48.795466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:48.797085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:48.807322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1134/2260] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:02:48.828507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0646d81951fa4353bdfb59048b498652'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:48.842826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0646d81951fa4353bdfb59048b498652'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:48.849556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0646d81951fa4353bdfb59048b498652'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:48.850199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0646d81951fa4353bdfb59048b498652'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:48.850814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a875398d-2d1b-4bdd-a772-70fe8ed3a10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0646d81951fa4353bdfb59048b498652'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:48.863381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1127/2261] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:02:48 2022] DELETE /identity/v3/users/0646d81951fa4353bdfb59048b498652 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:02:49.046011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.058059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc499e-ca61-4367-828e-e7522c7d2184 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.115957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.116588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.116865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.117653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] PATH_INFO: `/v3/projects/4d07d38e84224c9282dea262e712e4e3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.118753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d07d38e84224c9282dea262e712e4e3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.120697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.166047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d07d38e84224c9282dea262e712e4e3', 'request_id': 'req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:49.171932 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d07d38e84224c9282dea262e712e4e3', 'request_id': 'req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:49.177365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d07d38e84224c9282dea262e712e4e3', 'request_id': 'req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:49.180334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:49.191352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] The token cache is being invalidate because project 4d07d38e84224c9282dea262e712e4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:02:49.191642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:49.192273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bc7c8d74-00e3-4f77-88eb-ad23d1468bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:02:49.193836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1135/2262] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:02:49 2022] DELETE /identity/v3/projects/4d07d38e84224c9282dea262e712e4e3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:02:49.199896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630508ad-9f3f-4211-ac4e-2adf2129e7a9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.200178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630508ad-9f3f-4211-ac4e-2adf2129e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.200480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630508ad-9f3f-4211-ac4e-2adf2129e7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.219446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-630508ad-9f3f-4211-ac4e-2adf2129e7a9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:49.220667 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-630508ad-9f3f-4211-ac4e-2adf2129e7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:49.269426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:49.278963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1128/2263] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:49.286542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fcb4b-286c-494a-a860-833325eaa97b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.286921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fcb4b-286c-494a-a860-833325eaa97b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.287303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fcb4b-286c-494a-a860-833325eaa97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.308349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-452fcb4b-286c-494a-a860-833325eaa97b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:49.309547 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-452fcb4b-286c-494a-a860-833325eaa97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:49.317444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1136/2264] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:49.321758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630508ad-9f3f-4211-ac4e-2adf2129e7a9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.331293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630508ad-9f3f-4211-ac4e-2adf2129e7a9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.335796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.336418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.336694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.336963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.337600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:02:49.337899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.339470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.350507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1129/2265] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:02:49.355184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fcb4b-286c-494a-a860-833325eaa97b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.364335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fcb4b-286c-494a-a860-833325eaa97b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.369619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.370157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.370438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.370718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.371342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.373361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.400986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1137/2266] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:02:49 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:02:49.405562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.416015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b684d-f29b-4bbd-99e9-6f223c0afbd2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.471714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.472349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.472627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.472897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.474588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.477060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.499085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1130/2267] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:02:49 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:02:49.503189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.514187 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d554dc-ed50-4561-b8da-f684af72b6ae admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.520872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.521896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.522403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.522878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.524746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.527660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.538826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1138/2268] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:02:49.542984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.552915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a116d-9d24-48ca-820d-e79097fe3e42 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.557448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186fcb2-cc51-497e-bd69-75edf591f689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.558037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186fcb2-cc51-497e-bd69-75edf591f689 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.558312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186fcb2-cc51-497e-bd69-75edf591f689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.558583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186fcb2-cc51-497e-bd69-75edf591f689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.559213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186fcb2-cc51-497e-bd69-75edf591f689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.560846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186fcb2-cc51-497e-bd69-75edf591f689 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.571817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1131/2269] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:02:49.575811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.586525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214dca1-8017-49ec-8889-965671116e4c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.591174 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.591819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.592101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.592407 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] PATH_INFO: `/v3/projects/7351c123c59146cf80c23e0d1ffe80fb/users/6131a8d2e9374070bc770270c7a70519/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.606869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7351c123c59146cf80c23e0d1ffe80fb, user_id=6131a8d2e9374070bc770270c7a70519, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.609561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.622402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1139/2270] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:02:49 2022] PUT /identity/v3/projects/7351c123c59146cf80c23e0d1ffe80fb/users/6131a8d2e9374070bc770270c7a70519/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:02:49.627244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e3194-9be4-4c74-bafd-23594cb394eb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.627718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e3194-9be4-4c74-bafd-23594cb394eb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.627832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e3194-9be4-4c74-bafd-23594cb394eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.647355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8d7e3194-9be4-4c74-bafd-23594cb394eb None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:49.648613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8d7e3194-9be4-4c74-bafd-23594cb394eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:49.705237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1132/2271] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:49.715312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:49.717069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:49.774282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc156-2e4a-4704-a05a-129dfb7403a0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:49.777877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:49.778468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:49.778740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:49.779008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:49.787725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:49.789804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:49.798966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1140/2272] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:02:53.995765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee773418-2228-45fd-a3aa-7b8f365ffbf5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:53.996326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee773418-2228-45fd-a3aa-7b8f365ffbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:53.996591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee773418-2228-45fd-a3aa-7b8f365ffbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:54.032117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ee773418-2228-45fd-a3aa-7b8f365ffbf5 None None] MFA Rules not processed for user `6131a8d2e9374070bc770270c7a70519`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:54.033613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ee773418-2228-45fd-a3aa-7b8f365ffbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:54.096061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:54.107519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1133/2273] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:02:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:02:54.111832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:54.122426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8763aca-7da5-4ab1-b51b-d344617984d0 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:54.127166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:54.127774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:54.128066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:54.128376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:54.129007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:54.131295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:54.159487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1141/2274] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:02:54 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:02:54.163621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee773418-2228-45fd-a3aa-7b8f365ffbf5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:54.175319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee773418-2228-45fd-a3aa-7b8f365ffbf5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:54.233716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:54.234334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:54.234678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:54.234885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:54.235540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:54.237554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:54.261814 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1134/2275] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:02:54 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:02:54.263427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:54.275026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24518a0-89d5-4b88-bd68-bdb7bff1c104 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:54.280052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:54.280697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:54.281014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:54.281268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:54.281910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:54.283509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:54.304140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1142/2276] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:02:54.308628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:54.319006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf99f-1a29-4924-8a4c-817529b1a4cc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:54.323846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:54.324425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:54.324704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:54.324979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] PATH_INFO: `/v3/projects/7c5aad4db06146dbbd7036fff2667cd4/users/5a80616de4534b7f87e5f0573d077a6e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:54.341248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5aad4db06146dbbd7036fff2667cd4, user_id=5a80616de4534b7f87e5f0573d077a6e, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:54.343986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:54.358183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1135/2277] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:02:54 2022] PUT /identity/v3/projects/7c5aad4db06146dbbd7036fff2667cd4/users/5a80616de4534b7f87e5f0573d077a6e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:02:54.362210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:54.372619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceb1bc-fad4-443c-8dfb-e33ee6a45073 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:54.432930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eb295-81c6-4278-b965-66fc79495f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:54.433671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb295-81c6-4278-b965-66fc79495f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:54.433952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb295-81c6-4278-b965-66fc79495f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:54.434238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb295-81c6-4278-b965-66fc79495f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:54.434879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eb295-81c6-4278-b965-66fc79495f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:54.436516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eb295-81c6-4278-b965-66fc79495f0f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:54.437810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:54.447858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1143/2278] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:02:54.452205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:54.463344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65793af-4d76-4c04-ba46-822575c008eb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:54.468112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:54.468779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:54.469059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:54.469327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] PATH_INFO: `/v3/projects/7c5aad4db06146dbbd7036fff2667cd4/users/5a80616de4534b7f87e5f0573d077a6e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:54.471212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5aad4db06146dbbd7036fff2667cd4, user_id=5a80616de4534b7f87e5f0573d077a6e, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:54.473870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:54.486738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1136/2279] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:02:54 2022] PUT /identity/v3/projects/7c5aad4db06146dbbd7036fff2667cd4/users/5a80616de4534b7f87e5f0573d077a6e/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:02:59.251405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bcdf7-16aa-4849-bfe2-9c80a3df77f0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:59.251929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bcdf7-16aa-4849-bfe2-9c80a3df77f0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:59.251929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bcdf7-16aa-4849-bfe2-9c80a3df77f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:59.283758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4e4bcdf7-16aa-4849-bfe2-9c80a3df77f0 None None] MFA Rules not processed for user `5a80616de4534b7f87e5f0573d077a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:59.285011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4e4bcdf7-16aa-4849-bfe2-9c80a3df77f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:59.360348 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1144/2280] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:02:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:59.372584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:59.374318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:59.433080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b14cb-7e34-4e4f-abf5-9ba035a53d2e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:59.436608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:59.437230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:59.437507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:59.437775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:59.495152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:59.496712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:59.506619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1137/2281] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:02:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:02:59.654168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5598c-3b2e-4fba-9495-3724ce988351 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:59.654899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5598c-3b2e-4fba-9495-3724ce988351 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:59.655252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5598c-3b2e-4fba-9495-3724ce988351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:59.680781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cae5598c-3b2e-4fba-9495-3724ce988351 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:02:59.682069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cae5598c-3b2e-4fba-9495-3724ce988351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:59.771607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1145/2282] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 02:02:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:02:59.825172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:02:59.827284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:02:59.833686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:02:59.923542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f3233-da32-4a5d-b970-6595f2017881 service glance] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:02:59.930313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea764d84-66c6-4f82-aa9a-8da843c82bdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:02:59.931141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764d84-66c6-4f82-aa9a-8da843c82bdc service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:02:59.931497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764d84-66c6-4f82-aa9a-8da843c82bdc service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:02:59.931844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764d84-66c6-4f82-aa9a-8da843c82bdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:02:59.943334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea764d84-66c6-4f82-aa9a-8da843c82bdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:02:59.945603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea764d84-66c6-4f82-aa9a-8da843c82bdc service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:02:59.955607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1138/2283] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:02:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:03:02.840125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:02.840657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:02.840995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:02.866029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] MFA Rules not processed for user `86f364f03c7d4360b4a75ed194846ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:03:02.867879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:02.950325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1146/2284] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:03:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:03:09.047483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48ea0a-331e-40f1-8653-871dd0ba216c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:09.048064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48ea0a-331e-40f1-8653-871dd0ba216c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:09.048064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48ea0a-331e-40f1-8653-871dd0ba216c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:09.068073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ec48ea0a-331e-40f1-8653-871dd0ba216c None None] MFA Rules not processed for user `6131a8d2e9374070bc770270c7a70519`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:03:09.069331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ec48ea0a-331e-40f1-8653-871dd0ba216c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:09.076716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1139/2285] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:03:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:03:09.374666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:03:09.376337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:09.419054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:03:09.471252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afb1d2-c1ea-4629-a1ba-f780647cd73b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:03:09.477324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:03:09.477927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:09.478205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:09.478473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:09.480192 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:09.494025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:03:09.495639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:03:09.509037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1147/2286] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:03:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:03:14.074007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:14.074501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:14.074614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:14.096217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:03:14.097632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:14.107562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1140/2287] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 02:03:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:03:14.124278 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:03:14.139590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c22c82-5a90-4034-9f98-2ad94a2b85d1 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:03:14.146053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197b5d2-983f-43dd-9895-c49e73131c7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:03:14.146992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197b5d2-983f-43dd-9895-c49e73131c7e service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:14.147455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197b5d2-983f-43dd-9895-c49e73131c7e service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:14.148031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197b5d2-983f-43dd-9895-c49e73131c7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:14.165855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197b5d2-983f-43dd-9895-c49e73131c7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:03:14.168172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197b5d2-983f-43dd-9895-c49e73131c7e service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:03:14.183066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1148/2288] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:03:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:03:24.090192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=37 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:03:24.176178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=26 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:03:45.220397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:03:45.222485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:45.472163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2c2d-fef9-4a8b-a148-4b6f55eb8c51 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:03:45.478105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac3468-33a8-41df-88d3-994504f5560e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:03:45.478903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3468-33a8-41df-88d3-994504f5560e service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:45.479181 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3468-33a8-41df-88d3-994504f5560e service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:45.479452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3468-33a8-41df-88d3-994504f5560e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:45.481249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-07ac3468-33a8-41df-88d3-994504f5560e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:45.502035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac3468-33a8-41df-88d3-994504f5560e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:03:45.503860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac3468-33a8-41df-88d3-994504f5560e service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:03:45.521832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1141/2289] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:03:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:03:46.267232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:46.267782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:46.267782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:46.287809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] MFA Rules not processed for user `85d2f3d2a8cb40d38d0ca2dc16609ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:03:46.289333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:46.356520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1149/2290] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:03:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:03:48.912689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb13e0b-b3d7-42bb-864d-e523c37189fb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:48.913194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb13e0b-b3d7-42bb-864d-e523c37189fb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:48.913295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb13e0b-b3d7-42bb-864d-e523c37189fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:48.933288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2cb13e0b-b3d7-42bb-864d-e523c37189fb None None] MFA Rules not processed for user `a06de37941264b979a3bd5aaf84ec73c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:03:48.934709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2cb13e0b-b3d7-42bb-864d-e523c37189fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:48.990190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1142/2291] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:03:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:03:56.350771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=25 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:03:58.111264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:03:58.112706 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:03:58.190088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097d16e-07c7-4530-8a35-5040baf734e5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:03:58.193705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:03:58.194303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:03:58.194578 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:03:58.194851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:03:58.204148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:03:58.205721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:03:58.214969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1150/2292] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:03:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:03:58.983086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=88 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:08.210941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=35 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:47.105872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb13e0b-b3d7-42bb-864d-e523c37189fb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:47.115307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb13e0b-b3d7-42bb-864d-e523c37189fb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:47.118794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:47.119384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:47.119658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:47.119928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:47.121697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:47.206439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:47.207790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:47.226772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1143/2293] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:50.405251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:50.406676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:50.421709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f669d1-91e8-4b60-925a-81feef096a1f service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:50.426620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:50.427222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:50.427502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:50.427772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:50.432229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:50.444677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:50.446289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:50.457467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1151/2294] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:50.465399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:50.467712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:50.482212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195467f-77af-48dd-9290-8fc726b903f6 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:50.487483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:50.488284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:50.488428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:50.488697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] PATH_INFO: `/v3/users/86f364f03c7d4360b4a75ed194846ade` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:50.489806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f364f03c7d4360b4a75ed194846ade)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:50.491728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:50.539108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f364f03c7d4360b4a75ed194846ade'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.551382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f364f03c7d4360b4a75ed194846ade'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.558951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f364f03c7d4360b4a75ed194846ade'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.559571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f364f03c7d4360b4a75ed194846ade'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.560175 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-798c7c0e-c3ca-4c53-92de-e9f0a589def4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f364f03c7d4360b4a75ed194846ade'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.572683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1144/2295] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:04:50 2022] DELETE /identity/v3/users/86f364f03c7d4360b4a75ed194846ade => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:50.736711 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:50.747436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13853f-b47a-4a2d-884f-86d90a055efb service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:50.811817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:50.812488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:50.812749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:50.813020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] PATH_INFO: `/v3/projects/688f10cbc5ce42b895bb7a9285749cca` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:50.814128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=688f10cbc5ce42b895bb7a9285749cca)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:50.816026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:50.860656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688f10cbc5ce42b895bb7a9285749cca', 'request_id': 'req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.866778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688f10cbc5ce42b895bb7a9285749cca', 'request_id': 'req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.874622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688f10cbc5ce42b895bb7a9285749cca', 'request_id': 'req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.891823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] The token cache is being invalidate because project 688f10cbc5ce42b895bb7a9285749cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:04:50.892132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.893066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d93a55fa-b7dc-4472-8279-6a35cff4fa0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:50.894635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1152/2296] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:04:50 2022] DELETE /identity/v3/projects/688f10cbc5ce42b895bb7a9285749cca => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:50.899926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee839-f627-4bfa-904d-a938721a8182 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:50.900773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee839-f627-4bfa-904d-a938721a8182 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:50.901053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee839-f627-4bfa-904d-a938721a8182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:50.956805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:50.960010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b1dee839-f627-4bfa-904d-a938721a8182 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:50.961681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b1dee839-f627-4bfa-904d-a938721a8182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:51.022650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1145/2297] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:51.030978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76b7eb-fd95-421a-a6b5-0c3759fbfeb2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.031257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76b7eb-fd95-421a-a6b5-0c3759fbfeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.031529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76b7eb-fd95-421a-a6b5-0c3759fbfeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.051322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7f76b7eb-fd95-421a-a6b5-0c3759fbfeb2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:51.052524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7f76b7eb-fd95-421a-a6b5-0c3759fbfeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:51.060234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1153/2298] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:51.064326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dee839-f627-4bfa-904d-a938721a8182 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.074782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dee839-f627-4bfa-904d-a938721a8182 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.079289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.079875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.080149 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.080461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.081104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:04:51.081401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.082973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.094671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1146/2299] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:04:51.098703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76b7eb-fd95-421a-a6b5-0c3759fbfeb2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.107660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76b7eb-fd95-421a-a6b5-0c3759fbfeb2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.112233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.112847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.113120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.113388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.114007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.116001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.144307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1154/2300] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:04:51 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:51.148317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.158679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f3bb7-35b5-4356-8d57-390904ae9957 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.212562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.213777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.214062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.214336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.214968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.216911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.237876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1147/2301] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:04:51 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:51.242366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.253227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6e8a1-b4cb-4557-b97e-2ba015a7f0ce admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.258299 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.259293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.260203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.260605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.261412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.263088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.273687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1155/2302] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:51.278037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.292337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d2b00-5ec2-4c13-8c22-41dc93e12b2c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.297512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cdf69-a191-42fa-aedb-e02832a0abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.298137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cdf69-a191-42fa-aedb-e02832a0abcf admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.298420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cdf69-a191-42fa-aedb-e02832a0abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.298705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cdf69-a191-42fa-aedb-e02832a0abcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.299361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cdf69-a191-42fa-aedb-e02832a0abcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.301043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cdf69-a191-42fa-aedb-e02832a0abcf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.312346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1148/2303] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:51.316458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.327544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385af82-0799-4457-95d8-0822f78c22dc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.332517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.333191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.333464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.333735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] PATH_INFO: `/v3/projects/81555303c6454d618822b3a29a4f32a1/users/2b627bf60fbb4ef295c14008b4c8f615/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.350097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81555303c6454d618822b3a29a4f32a1, user_id=2b627bf60fbb4ef295c14008b4c8f615, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.352761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.365963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1156/2304] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:51 2022] PUT /identity/v3/projects/81555303c6454d618822b3a29a4f32a1/users/2b627bf60fbb4ef295c14008b4c8f615/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:51.376230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423ceb5-efa2-4c7d-9945-4f3ffee4d12e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.376562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423ceb5-efa2-4c7d-9945-4f3ffee4d12e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.376891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423ceb5-efa2-4c7d-9945-4f3ffee4d12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.411379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9423ceb5-efa2-4c7d-9945-4f3ffee4d12e None None] MFA Rules not processed for user `2b627bf60fbb4ef295c14008b4c8f615`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:51.412660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9423ceb5-efa2-4c7d-9945-4f3ffee4d12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:51.475696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1149/2305] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:51 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:51.480109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.493135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:51.543982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697fad7-b145-4963-b043-ba559316933d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.549243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.549845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.550124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.550394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.551020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.552971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.580773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1157/2306] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:04:51 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:51.585116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423ceb5-efa2-4c7d-9945-4f3ffee4d12e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.590981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:51.597815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423ceb5-efa2-4c7d-9945-4f3ffee4d12e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.654197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.654744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.655020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.655291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.655925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.657880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.677894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1150/2307] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:04:51 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:51.682608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.692752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80444fe-5915-4425-905f-4e39c1d43975 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.698050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.698670 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.698949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.699221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.699857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.701488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.711736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1158/2308] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:51.715737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.725695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed60e-8d72-42c5-ae6b-d7c517a0bd5f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.730221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.730853 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.731129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.731401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] PATH_INFO: `/v3/projects/88d0a36594384f81b1b07a7c5e92bd96/users/74236cbd88ca448b8bbf46d9ff1c1722/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.745921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d0a36594384f81b1b07a7c5e92bd96, user_id=74236cbd88ca448b8bbf46d9ff1c1722, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.748706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.763262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1151/2309] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:51 2022] PUT /identity/v3/projects/88d0a36594384f81b1b07a7c5e92bd96/users/74236cbd88ca448b8bbf46d9ff1c1722/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:51.767104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.777045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52711b-989d-4dd8-bb31-bd72d3a2f8dc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.831817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45a684-624b-4c29-8d44-d63c84fe527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.832444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45a684-624b-4c29-8d44-d63c84fe527d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.832724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45a684-624b-4c29-8d44-d63c84fe527d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.832997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45a684-624b-4c29-8d44-d63c84fe527d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.833627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45a684-624b-4c29-8d44-d63c84fe527d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.835217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45a684-624b-4c29-8d44-d63c84fe527d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.844604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1159/2310] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:51.848855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:51.859721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f794e3d-b3a6-4cbf-9b6f-19666ff746ab admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:51.864392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:51.865035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.865341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.865613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] PATH_INFO: `/v3/projects/88d0a36594384f81b1b07a7c5e92bd96/users/74236cbd88ca448b8bbf46d9ff1c1722/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.867507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d0a36594384f81b1b07a7c5e92bd96, user_id=74236cbd88ca448b8bbf46d9ff1c1722, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:51.870744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:51.880617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1152/2311] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:51 2022] PUT /identity/v3/projects/88d0a36594384f81b1b07a7c5e92bd96/users/74236cbd88ca448b8bbf46d9ff1c1722/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:51.892396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac63932-ae98-4504-a10b-493a6b549bc7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:51.892691 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac63932-ae98-4504-a10b-493a6b549bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:51.892968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac63932-ae98-4504-a10b-493a6b549bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:51.921047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0ac63932-ae98-4504-a10b-493a6b549bc7 None None] MFA Rules not processed for user `74236cbd88ca448b8bbf46d9ff1c1722`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:51.922250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0ac63932-ae98-4504-a10b-493a6b549bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:51.990443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1160/2312] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:04:52.001094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:52.002766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:52.080829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a933e-6e6e-4ac5-860d-b730cf4305fe admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:52.084487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae054f0e-b20f-4b97-9a68-3fb62791ffd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:52.085076 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae054f0e-b20f-4b97-9a68-3fb62791ffd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:52.085355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae054f0e-b20f-4b97-9a68-3fb62791ffd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:52.085626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae054f0e-b20f-4b97-9a68-3fb62791ffd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:52.095034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae054f0e-b20f-4b97-9a68-3fb62791ffd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:52.096477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae054f0e-b20f-4b97-9a68-3fb62791ffd9 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:52.105773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1153/2313] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:52 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:52.252030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac63932-ae98-4504-a10b-493a6b549bc7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:52.262387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:52.315200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac63932-ae98-4504-a10b-493a6b549bc7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:52.319668 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:52.320311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:52.320590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:52.320861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] PATH_INFO: `/v3/users/2b627bf60fbb4ef295c14008b4c8f615` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:52.321964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b627bf60fbb4ef295c14008b4c8f615)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:52.324515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:52.371779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b627bf60fbb4ef295c14008b4c8f615'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.378787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b627bf60fbb4ef295c14008b4c8f615'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.386616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b627bf60fbb4ef295c14008b4c8f615'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.387444 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b627bf60fbb4ef295c14008b4c8f615'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.399430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b627bf60fbb4ef295c14008b4c8f615'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.405811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1161/2314] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:04:52 2022] DELETE /identity/v3/users/2b627bf60fbb4ef295c14008b4c8f615 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:52.410356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:52.410661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:52.410964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:52.430472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:52.431636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:52.462127 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:52.490499 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1154/2315] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:52.500094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:52.501543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:52.557930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4adb66-e3ed-4dfb-8397-2445da857881 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:52.561644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:52.562238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:52.562516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:52.562789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:52.571678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:52.573659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:52.582880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1162/2316] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:52.850535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:52.851962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:52.866129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2764ab-ed32-4142-aa63-fc1b15614596 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:52.871632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:52.872276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:52.872578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:52.872900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] PATH_INFO: `/v3/projects/81555303c6454d618822b3a29a4f32a1` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:52.874047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81555303c6454d618822b3a29a4f32a1)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:52.875906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:52.922969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81555303c6454d618822b3a29a4f32a1', 'request_id': 'req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.930477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81555303c6454d618822b3a29a4f32a1', 'request_id': 'req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.937631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81555303c6454d618822b3a29a4f32a1', 'request_id': 'req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.950022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] The token cache is being invalidate because project 81555303c6454d618822b3a29a4f32a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:04:52.950307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.950954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:52.952673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1155/2317] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:04:52 2022] DELETE /identity/v3/projects/81555303c6454d618822b3a29a4f32a1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:52.956458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:52.965912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd352d1-9cdc-49c8-9b3e-29e190ea9192 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:52.993518 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:53.022722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.023337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.023619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.023892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] PATH_INFO: `/v3/users/74236cbd88ca448b8bbf46d9ff1c1722` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.025086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74236cbd88ca448b8bbf46d9ff1c1722)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:53.027006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:53.071352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74236cbd88ca448b8bbf46d9ff1c1722'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.078724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74236cbd88ca448b8bbf46d9ff1c1722'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.085795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74236cbd88ca448b8bbf46d9ff1c1722'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.086433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74236cbd88ca448b8bbf46d9ff1c1722'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.098141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bdd494a5-830c-485c-aeb8-aaca9f7c6a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74236cbd88ca448b8bbf46d9ff1c1722'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.103982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1163/2318] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:04:52 2022] DELETE /identity/v3/users/74236cbd88ca448b8bbf46d9ff1c1722 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:53.397348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:53.400755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:53.417371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31e6ef-789f-40e0-a8f5-46ba50ca9711 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:53.480698 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.481331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.481614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.481885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] PATH_INFO: `/v3/projects/88d0a36594384f81b1b07a7c5e92bd96` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.482997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88d0a36594384f81b1b07a7c5e92bd96)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:53.484922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:53.531493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88d0a36594384f81b1b07a7c5e92bd96', 'request_id': 'req-89a4400a-e5d0-4a71-902d-266d56c7de27'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.538895 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88d0a36594384f81b1b07a7c5e92bd96', 'request_id': 'req-89a4400a-e5d0-4a71-902d-266d56c7de27'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.544097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:53.545515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88d0a36594384f81b1b07a7c5e92bd96', 'request_id': 'req-89a4400a-e5d0-4a71-902d-266d56c7de27'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.560158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] The token cache is being invalidate because project 88d0a36594384f81b1b07a7c5e92bd96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:04:53.560485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.561136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-89a4400a-e5d0-4a71-902d-266d56c7de27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:53.562879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1156/2319] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:04:53 2022] DELETE /identity/v3/projects/88d0a36594384f81b1b07a7c5e92bd96 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:53.567571 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32149967-d51b-4d52-a850-af14b6f27fe8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.568036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32149967-d51b-4d52-a850-af14b6f27fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.568168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32149967-d51b-4d52-a850-af14b6f27fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.587257 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-32149967-d51b-4d52-a850-af14b6f27fe8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:53.588505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-32149967-d51b-4d52-a850-af14b6f27fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:53.649669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1164/2320] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:53.657697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2ddc-8a58-4d92-9841-cc321cf98210 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.657987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2ddc-8a58-4d92-9841-cc321cf98210 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.658267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2ddc-8a58-4d92-9841-cc321cf98210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.679786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6e7a2ddc-8a58-4d92-9841-cc321cf98210 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:53.681062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6e7a2ddc-8a58-4d92-9841-cc321cf98210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:53.689740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1157/2321] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:53.694327 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32149967-d51b-4d52-a850-af14b6f27fe8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:53.704574 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32149967-d51b-4d52-a850-af14b6f27fe8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:53.709376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.710070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.710417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.710757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.711486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:04:53.711860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:53.714085 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:53.730621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1165/2322] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:04:53.731684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2ddc-8a58-4d92-9841-cc321cf98210 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:53.741577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2ddc-8a58-4d92-9841-cc321cf98210 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:53.746527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.747148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.747430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.747705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.748422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:53.750340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:53.784576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1158/2323] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:04:53 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:04:53.785604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:53.796555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cead4-bc29-4caa-8b20-fdff48ffdea6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:53.829741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:53.856980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.857596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.857878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.858178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.858780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:53.861071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:53.888280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1166/2324] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:04:53 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:04:53.890165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:53.900865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d843f-5b19-458f-aa24-25398687cf44 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:53.905598 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.906190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.906467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.906872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.907788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:53.909686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:53.923507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1159/2325] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:53.924836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:53.935501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5de96-a4d2-468e-a1e2-35b08346174f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:53.945309 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641c65f-456c-4417-84c2-1cd9d252dabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.946091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641c65f-456c-4417-84c2-1cd9d252dabc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.946371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641c65f-456c-4417-84c2-1cd9d252dabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.946617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641c65f-456c-4417-84c2-1cd9d252dabc admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.947274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641c65f-456c-4417-84c2-1cd9d252dabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:53.949080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641c65f-456c-4417-84c2-1cd9d252dabc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:53.961348 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1167/2326] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:53.965873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:53.976190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1080cbe-e876-4db2-9ff7-c584e11c43de admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:53.981572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:53.982294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:53.982637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:53.982913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] PATH_INFO: `/v3/projects/8d6b9e01ffdb4537813bb4d739fb5264/users/2c56498586ee4121a2e6634b1106eb62/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:53.997360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d6b9e01ffdb4537813bb4d739fb5264, user_id=2c56498586ee4121a2e6634b1106eb62, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.001426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.015345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1160/2327] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:53 2022] PUT /identity/v3/projects/8d6b9e01ffdb4537813bb4d739fb5264/users/2c56498586ee4121a2e6634b1106eb62/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:54.026105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e72b26-d7e0-450c-afff-f5270d4f8591 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.026388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e72b26-d7e0-450c-afff-f5270d4f8591 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.026658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e72b26-d7e0-450c-afff-f5270d4f8591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.054744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e7e72b26-d7e0-450c-afff-f5270d4f8591 None None] MFA Rules not processed for user `2c56498586ee4121a2e6634b1106eb62`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:54.055841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e7e72b26-d7e0-450c-afff-f5270d4f8591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:54.121557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1168/2328] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:04:54.125870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:54.191242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb267dd6-1e9c-4a02-b26f-3894a7aed975 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:54.196194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:54.197594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.197868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.198133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.199143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.201098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.229467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1161/2329] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:04:54 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:54.233703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e72b26-d7e0-450c-afff-f5270d4f8591 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:54.245220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e72b26-d7e0-450c-afff-f5270d4f8591 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:54.306382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:54.306918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.307190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.307458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.308083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.310027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.339657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1169/2330] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:04:54 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:54.341924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:54.352960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e2fc6-90bb-4723-b288-45826d942831 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:54.358065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:54.358679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.359357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.359357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.359910 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.361770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.378274 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1162/2331] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:54.379574 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:54.392135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c7a3a-5164-49dc-af3f-c31d2687e9a7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:54.398841 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:54.399505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.399778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.400076 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] PATH_INFO: `/v3/projects/22558f23cae14291a6a51880b269a138/users/9d1075e3321441b897ff85a54c76cf1c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.420207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22558f23cae14291a6a51880b269a138, user_id=9d1075e3321441b897ff85a54c76cf1c, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.423193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.440274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1170/2332] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:54 2022] PUT /identity/v3/projects/22558f23cae14291a6a51880b269a138/users/9d1075e3321441b897ff85a54c76cf1c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:54.444873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:54.460656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe0dfb-4734-49b2-8844-71685648745b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:54.559893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c5f0f-0ef8-4edc-a07d-a1059b007ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:54.560607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c5f0f-0ef8-4edc-a07d-a1059b007ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.560900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c5f0f-0ef8-4edc-a07d-a1059b007ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.561292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c5f0f-0ef8-4edc-a07d-a1059b007ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.562001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c5f0f-0ef8-4edc-a07d-a1059b007ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.564411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c5f0f-0ef8-4edc-a07d-a1059b007ce3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.566545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:54.582619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1163/2333] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:54.586337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:54.597475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee644d-ad7f-4c4f-9ab2-f2e9fba913d4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:54.602637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:54.603550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.603800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.604140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] PATH_INFO: `/v3/projects/22558f23cae14291a6a51880b269a138/users/9d1075e3321441b897ff85a54c76cf1c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.606230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22558f23cae14291a6a51880b269a138, user_id=9d1075e3321441b897ff85a54c76cf1c, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.609232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.615848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:54.628305 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1171/2334] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:54 2022] PUT /identity/v3/projects/22558f23cae14291a6a51880b269a138/users/9d1075e3321441b897ff85a54c76cf1c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:54.639010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f686124-5c31-4c64-b8ef-a4e1e14b75ea None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.639987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f686124-5c31-4c64-b8ef-a4e1e14b75ea None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.640647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f686124-5c31-4c64-b8ef-a4e1e14b75ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.675768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5f686124-5c31-4c64-b8ef-a4e1e14b75ea None None] MFA Rules not processed for user `9d1075e3321441b897ff85a54c76cf1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:54.677169 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5f686124-5c31-4c64-b8ef-a4e1e14b75ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:54.760381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1164/2335] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:04:54.772037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:54.773813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:54.832491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7868c9-2bd1-4351-abf9-f8ae22a6510a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:54.836170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a83da-bd71-458f-8105-99b88680f02f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:54.836807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a83da-bd71-458f-8105-99b88680f02f service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:54.837086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a83da-bd71-458f-8105-99b88680f02f service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:54.837357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a83da-bd71-458f-8105-99b88680f02f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:54.846656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a83da-bd71-458f-8105-99b88680f02f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:54.848079 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a83da-bd71-458f-8105-99b88680f02f service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:54.858230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1172/2336] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:55.255045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f686124-5c31-4c64-b8ef-a4e1e14b75ea None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:55.326528 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f686124-5c31-4c64-b8ef-a4e1e14b75ea None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:55.331208 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:55.331851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:55.332156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:55.332461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] PATH_INFO: `/v3/users/2c56498586ee4121a2e6634b1106eb62` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:55.333631 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c56498586ee4121a2e6634b1106eb62)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:55.335562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:55.384100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c56498586ee4121a2e6634b1106eb62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:55.395968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c56498586ee4121a2e6634b1106eb62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:55.404550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c56498586ee4121a2e6634b1106eb62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:55.405291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c56498586ee4121a2e6634b1106eb62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:55.405886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c56498586ee4121a2e6634b1106eb62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:55.420332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1165/2337] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:04:55 2022] DELETE /identity/v3/users/2c56498586ee4121a2e6634b1106eb62 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:55.425138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:55.425491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:55.425830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:55.445443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:55.446733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:55.507605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1173/2338] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:55.517581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:55.518985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:55.524400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:55.577826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12329fe-418a-4b35-a2c0-bcf45d32314f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:55.581404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:55.581996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:55.582272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:55.582541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:55.591760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:55.593185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:55.602307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1166/2339] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:55.967029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:55.968871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:55.995273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5349f-d3a2-421e-bdda-eacd988a3582 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:56.001011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:56.001698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.002207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.002679 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] PATH_INFO: `/v3/projects/8d6b9e01ffdb4537813bb4d739fb5264` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.003851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d6b9e01ffdb4537813bb4d739fb5264)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:56.005842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:56.057943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d6b9e01ffdb4537813bb4d739fb5264', 'request_id': 'req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.064025 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d6b9e01ffdb4537813bb4d739fb5264', 'request_id': 'req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.069696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d6b9e01ffdb4537813bb4d739fb5264', 'request_id': 'req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.081685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] The token cache is being invalidate because project 8d6b9e01ffdb4537813bb4d739fb5264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:04:56.081970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.082588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.084121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1174/2340] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:04:55 2022] DELETE /identity/v3/projects/8d6b9e01ffdb4537813bb4d739fb5264 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:56.088311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:56.097989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf24b4-e6e8-4521-9899-85a4be49b7ec service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:56.154808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:56.155453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.155729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.156045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] PATH_INFO: `/v3/users/9d1075e3321441b897ff85a54c76cf1c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.157244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1075e3321441b897ff85a54c76cf1c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:56.159247 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:56.203468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1075e3321441b897ff85a54c76cf1c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.215285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1075e3321441b897ff85a54c76cf1c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.222292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1075e3321441b897ff85a54c76cf1c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.222955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1075e3321441b897ff85a54c76cf1c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.223562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c8abe96c-4e5c-4560-bcaa-a2825d26ddfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1075e3321441b897ff85a54c76cf1c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.236366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1167/2341] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:04:56 2022] DELETE /identity/v3/users/9d1075e3321441b897ff85a54c76cf1c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:56.509504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:56.510983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:56.526373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1714-4b9e-4d58-bc77-cbb2799789a2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:56.547631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:56.585819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:56.586450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.586726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.586999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] PATH_INFO: `/v3/projects/22558f23cae14291a6a51880b269a138` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.588105 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22558f23cae14291a6a51880b269a138)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:56.590030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:56.634776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22558f23cae14291a6a51880b269a138', 'request_id': 'req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.640626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22558f23cae14291a6a51880b269a138', 'request_id': 'req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.645993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22558f23cae14291a6a51880b269a138', 'request_id': 'req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.658757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] The token cache is being invalidate because project 22558f23cae14291a6a51880b269a138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:04:56.659042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.659727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3ccb3704-aab8-4b6e-9125-37e7b9a6c214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:04:56.661306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1175/2342] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:04:56 2022] DELETE /identity/v3/projects/22558f23cae14291a6a51880b269a138 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:56.667839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee819b4b-7985-43a0-9cad-8fb059b745f2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.668335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee819b4b-7985-43a0-9cad-8fb059b745f2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.668450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee819b4b-7985-43a0-9cad-8fb059b745f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.688356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ee819b4b-7985-43a0-9cad-8fb059b745f2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:56.689476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ee819b4b-7985-43a0-9cad-8fb059b745f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:56.708087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:56.751062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1168/2343] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:56.759293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b68a3-320d-4534-8870-7d55d731f397 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.760019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b68a3-320d-4534-8870-7d55d731f397 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.760352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b68a3-320d-4534-8870-7d55d731f397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.780173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5a7b68a3-320d-4534-8870-7d55d731f397 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:56.781391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5a7b68a3-320d-4534-8870-7d55d731f397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:56.789221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1176/2344] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:56.794190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee819b4b-7985-43a0-9cad-8fb059b745f2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:56.803820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee819b4b-7985-43a0-9cad-8fb059b745f2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:56.808446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:56.809032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.809306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.809580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.810231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:04:56.810553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:56.812184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:56.823183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1169/2345] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:04:56.827334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b68a3-320d-4534-8870-7d55d731f397 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:56.836639 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b68a3-320d-4534-8870-7d55d731f397 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:56.842350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:56.842972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.843251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.843523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.844165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:56.846094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:56.876520 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1177/2346] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:04:56 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:56.877742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:56.887999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e53b8-e40b-46ea-85a0-57bbd9a65ff4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:56.944121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:56.944715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.945002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.945280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.945922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:56.947911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:56.969584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1170/2347] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:04:56 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:56.973834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:56.984788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f2836-d942-4069-af2f-0c60597324b5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:56.990538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:56.991151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:56.991431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:56.991707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:56.992715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:56.994649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.006092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1178/2348] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:57.010997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.022070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f98fc8-2168-46b3-b3f4-e9222231bdfd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.029168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74856-1025-4992-86f0-735f3aa5de5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.029956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74856-1025-4992-86f0-735f3aa5de5c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.030320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74856-1025-4992-86f0-735f3aa5de5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.030608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74856-1025-4992-86f0-735f3aa5de5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.031263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74856-1025-4992-86f0-735f3aa5de5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.033139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74856-1025-4992-86f0-735f3aa5de5c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.045199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1171/2349] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:04:57.049357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.060410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747b70c-34a5-428b-b6bd-3ca0ab34af53 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.065361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.066052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.066326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.066596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] PATH_INFO: `/v3/projects/1104986ed5b94f5a9c3b9816601e5ae1/users/7c773765ec624cfab7bc4e1b94d3553d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.083076 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1104986ed5b94f5a9c3b9816601e5ae1, user_id=7c773765ec624cfab7bc4e1b94d3553d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.085786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.101028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1179/2350] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:57 2022] PUT /identity/v3/projects/1104986ed5b94f5a9c3b9816601e5ae1/users/7c773765ec624cfab7bc4e1b94d3553d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:57.111357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8741f-85bb-461b-86bd-18aa6def936a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.111732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8741f-85bb-461b-86bd-18aa6def936a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.112076 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8741f-85bb-461b-86bd-18aa6def936a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.140815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3cd8741f-85bb-461b-86bd-18aa6def936a None None] MFA Rules not processed for user `7c773765ec624cfab7bc4e1b94d3553d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:57.142030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3cd8741f-85bb-461b-86bd-18aa6def936a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:57.209827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1172/2351] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:57.214390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.271882 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:57.287052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a524cc-3b81-4aee-a870-9a767cc3aa13 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.297098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.297716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.298041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.298742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.298976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.301031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.331246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1180/2352] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:04:57 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:04:57.335399 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8741f-85bb-461b-86bd-18aa6def936a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.347945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8741f-85bb-461b-86bd-18aa6def936a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.367119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:57.416788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.417551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.417675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.417920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.419388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.421817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.446864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1173/2353] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:04:57 2022] POST /identity/v3/users => generated 306 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:04:57.451370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.465288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e94b3-47ce-4348-b260-59d9f1aa5730 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.473595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.474243 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.474575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.474777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.475510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.477273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.489491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1181/2354] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:57.493645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.505212 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded55008-6911-4823-8c6a-13b3216c7249 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.510363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.511084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.511363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.511638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] PATH_INFO: `/v3/projects/15a0806d0eb34c8784d37ddfc18fcb9d/users/b13023dbc5424a0684b38281263dd217/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.526703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a0806d0eb34c8784d37ddfc18fcb9d, user_id=b13023dbc5424a0684b38281263dd217, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.529381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.542687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1174/2355] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:57 2022] PUT /identity/v3/projects/15a0806d0eb34c8784d37ddfc18fcb9d/users/b13023dbc5424a0684b38281263dd217/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:04:57.547197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.557955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9f596-848b-4259-89f1-e28c9bca6012 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.614763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96eee0-3a55-430e-a668-4a3fcf0a10e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.615359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96eee0-3a55-430e-a668-4a3fcf0a10e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.615633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96eee0-3a55-430e-a668-4a3fcf0a10e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.615902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96eee0-3a55-430e-a668-4a3fcf0a10e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.616560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96eee0-3a55-430e-a668-4a3fcf0a10e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.618152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96eee0-3a55-430e-a668-4a3fcf0a10e7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.627390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1182/2356] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:04:57.631623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.642006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ee448-97f0-43f6-b05a-03fa2555d913 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.646562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.647202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.647480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.647753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] PATH_INFO: `/v3/projects/15a0806d0eb34c8784d37ddfc18fcb9d/users/b13023dbc5424a0684b38281263dd217/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.649716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a0806d0eb34c8784d37ddfc18fcb9d, user_id=b13023dbc5424a0684b38281263dd217, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.652862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.662985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1175/2357] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:04:57 2022] PUT /identity/v3/projects/15a0806d0eb34c8784d37ddfc18fcb9d/users/b13023dbc5424a0684b38281263dd217/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:04:57.674703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116d679-6f50-4f94-a42e-16d5c68488d9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.674990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116d679-6f50-4f94-a42e-16d5c68488d9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.675261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116d679-6f50-4f94-a42e-16d5c68488d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.702190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0116d679-6f50-4f94-a42e-16d5c68488d9 None None] MFA Rules not processed for user `b13023dbc5424a0684b38281263dd217`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:04:57.703367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0116d679-6f50-4f94-a42e-16d5c68488d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:57.791417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1183/2358] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:04:57 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:04:57.804060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.805987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:57.868481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff2d79-5f83-4c39-a038-f3051cc56ff5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.873481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.874120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.874395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.874665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:57.885905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:57.887489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:57.897640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1176/2359] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:57 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:57.979492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116d679-6f50-4f94-a42e-16d5c68488d9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:57.991211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116d679-6f50-4f94-a42e-16d5c68488d9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:57.995658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:57.996306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:57.996586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:57.996858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:58.043848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:58.058307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:58.059751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:58.069310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1184/2360] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:57 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:04:59.644607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:04:59.645938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:59.706082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28fcbc-5031-4c26-b162-74dd321b351d service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:04:59.709839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54cbd6-938e-481c-a8ba-b93b4771c8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:04:59.710437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54cbd6-938e-481c-a8ba-b93b4771c8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:04:59.710770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54cbd6-938e-481c-a8ba-b93b4771c8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:04:59.711042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54cbd6-938e-481c-a8ba-b93b4771c8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:04:59.713383 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ed54cbd6-938e-481c-a8ba-b93b4771c8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:04:59.744119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:04:59.780416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed54cbd6-938e-481c-a8ba-b93b4771c8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:04:59.781856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed54cbd6-938e-481c-a8ba-b93b4771c8a2 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:04:59.792572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1177/2361] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:05:02.199230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:02.211579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b791212-2549-456d-98b7-b140b0c92081 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:02.216013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:02.216687 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:02.217033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:02.217318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] PATH_INFO: `/v3/users/7c773765ec624cfab7bc4e1b94d3553d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:02.218478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c773765ec624cfab7bc4e1b94d3553d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:02.220450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:02.286757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c773765ec624cfab7bc4e1b94d3553d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:02.304295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c773765ec624cfab7bc4e1b94d3553d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:02.314350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c773765ec624cfab7bc4e1b94d3553d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:02.315074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c773765ec624cfab7bc4e1b94d3553d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:02.331047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c773765ec624cfab7bc4e1b94d3553d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:02.340588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1185/2362] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:02 2022] DELETE /identity/v3/users/7c773765ec624cfab7bc4e1b94d3553d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:02.345111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:02.345969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:02.346251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:02.367914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:02.369774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:02.440715 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1178/2363] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:02.447893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:02.449364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:02.510780 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa277a-2373-46d7-a6ae-7eec2df7c043 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:02.514429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:02.515031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:02.515301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:02.515567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:02.526872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:02.528330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:02.539190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1186/2364] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:02.827178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:02.829032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:02.850477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d9538-e2ff-4766-b653-a17f26bcd7e2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:02.861790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:02.862682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:02.863097 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:02.863322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] PATH_INFO: `/v3/projects/1104986ed5b94f5a9c3b9816601e5ae1` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:02.865322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1104986ed5b94f5a9c3b9816601e5ae1)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:02.869664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:02.969343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1104986ed5b94f5a9c3b9816601e5ae1', 'request_id': 'req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:02.990280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1104986ed5b94f5a9c3b9816601e5ae1', 'request_id': 'req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.017712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1104986ed5b94f5a9c3b9816601e5ae1', 'request_id': 'req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.034252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] The token cache is being invalidate because project 1104986ed5b94f5a9c3b9816601e5ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:03.034541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.035201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.041617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1179/2365] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:02 2022] DELETE /identity/v3/projects/1104986ed5b94f5a9c3b9816601e5ae1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:03.042850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:03.052440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0223d8-cecd-443f-b09e-3cd327ec9e0e service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:03.116640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:03.117451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:03.117788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:03.118073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] PATH_INFO: `/v3/users/b13023dbc5424a0684b38281263dd217` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:03.119795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13023dbc5424a0684b38281263dd217)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:03.122128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:03.124227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:03.192109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13023dbc5424a0684b38281263dd217'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.200157 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13023dbc5424a0684b38281263dd217'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.208607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13023dbc5424a0684b38281263dd217'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.209236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13023dbc5424a0684b38281263dd217'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.240415 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-09760918-9275-4413-9492-70ac2f5d7c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13023dbc5424a0684b38281263dd217'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.248480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1187/2366] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:03 2022] DELETE /identity/v3/users/b13023dbc5424a0684b38281263dd217 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:03.709019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:03.710724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:03.737441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0febcdb-597b-4f7b-ba40-15bc59cf89c1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:03.828924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:03.861846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:03.862804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:03.863593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:03.864269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] PATH_INFO: `/v3/projects/15a0806d0eb34c8784d37ddfc18fcb9d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:03.865949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a0806d0eb34c8784d37ddfc18fcb9d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:03.869359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:03.966879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a0806d0eb34c8784d37ddfc18fcb9d', 'request_id': 'req-7f74d7b6-478e-4430-b214-53c457e87627'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.981246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a0806d0eb34c8784d37ddfc18fcb9d', 'request_id': 'req-7f74d7b6-478e-4430-b214-53c457e87627'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:03.988577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a0806d0eb34c8784d37ddfc18fcb9d', 'request_id': 'req-7f74d7b6-478e-4430-b214-53c457e87627'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:04.008141 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] The token cache is being invalidate because project 15a0806d0eb34c8784d37ddfc18fcb9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:04.011843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:04.012576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7f74d7b6-478e-4430-b214-53c457e87627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:04.014301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1180/2367] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:03 2022] DELETE /identity/v3/projects/15a0806d0eb34c8784d37ddfc18fcb9d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:04.019400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777548d-5b6d-4145-8bba-7365d48d3c85 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.019736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777548d-5b6d-4145-8bba-7365d48d3c85 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.019962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777548d-5b6d-4145-8bba-7365d48d3c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.088324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6777548d-5b6d-4145-8bba-7365d48d3c85 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:04.090616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6777548d-5b6d-4145-8bba-7365d48d3c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:04.204697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1188/2368] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:05:04.206524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c33e8a-8cc2-4a1f-be8c-6e57744124e2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.206818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c33e8a-8cc2-4a1f-be8c-6e57744124e2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.207101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c33e8a-8cc2-4a1f-be8c-6e57744124e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.234471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e3c33e8a-8cc2-4a1f-be8c-6e57744124e2 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:04.236416 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e3c33e8a-8cc2-4a1f-be8c-6e57744124e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:04.247015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1181/2369] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:04.250961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777548d-5b6d-4145-8bba-7365d48d3c85 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:04.263108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777548d-5b6d-4145-8bba-7365d48d3c85 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:04.270387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:04.271217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.271508 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.271847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.272917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:05:04.273372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:04.275225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:04.294625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1189/2370] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:05:04.299327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c33e8a-8cc2-4a1f-be8c-6e57744124e2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:04.331780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c33e8a-8cc2-4a1f-be8c-6e57744124e2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:04.337292 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:04.338387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.338755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.339213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.339826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:04.341942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:04.384327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1182/2371] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:05:04 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:05:04.388107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:04.399463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ea8cd-500e-4703-b29d-f13c355feeb3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:04.475316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:04.475930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.476310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.476650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.477369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:04.479445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:04.491834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:04.508807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1190/2372] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:05:04 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:05:04.513160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:04.523645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf1a9d-02e0-4d47-9686-3f37662d8c09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:04.528668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:04.529283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.529561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.529834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.530479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:04.532663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:04.548138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1183/2373] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:04.551991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:04.562878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f59a7b-c40b-48a5-ab90-86e076680c06 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:04.567718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bb97c-f740-44a7-9ab3-72537b33297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:04.568345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bb97c-f740-44a7-9ab3-72537b33297d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.568626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bb97c-f740-44a7-9ab3-72537b33297d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.568898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bb97c-f740-44a7-9ab3-72537b33297d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.569537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bb97c-f740-44a7-9ab3-72537b33297d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:04.571138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bb97c-f740-44a7-9ab3-72537b33297d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:04.582685 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1191/2374] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:04.587093 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:04.597240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecb631-b57a-4ed5-bbe0-664811c6a8d5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:04.601959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:04.602605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.602882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.603672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] PATH_INFO: `/v3/projects/70f27a79cbba4b0ba02139e1f5ee193c/users/9102e895cbad460cb51e57416e705474/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.618030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f27a79cbba4b0ba02139e1f5ee193c, user_id=9102e895cbad460cb51e57416e705474, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:04.621436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:04.635262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1184/2375] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:05:04 2022] PUT /identity/v3/projects/70f27a79cbba4b0ba02139e1f5ee193c/users/9102e895cbad460cb51e57416e705474/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:04.640581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aa033-5b82-4d29-82be-cda29edd1d4e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.640948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aa033-5b82-4d29-82be-cda29edd1d4e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.641303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aa033-5b82-4d29-82be-cda29edd1d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.661173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d29aa033-5b82-4d29-82be-cda29edd1d4e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:04.662400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d29aa033-5b82-4d29-82be-cda29edd1d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:04.724448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1192/2376] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:04.734720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:04.736443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:04.793075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61f6a8-52bc-446a-815f-1805aeeae661 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:04.796712 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3ab36-a418-4efd-935c-941cdd7f1353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:04.797311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ab36-a418-4efd-935c-941cdd7f1353 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:04.797586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ab36-a418-4efd-935c-941cdd7f1353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:04.797855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ab36-a418-4efd-935c-941cdd7f1353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:04.807237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3ab36-a418-4efd-935c-941cdd7f1353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:04.808737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3ab36-a418-4efd-935c-941cdd7f1353 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:04.818211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1185/2377] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:05.217914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024716a1-f2c5-4d3c-8f78-42daaecc20da None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:05.218417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024716a1-f2c5-4d3c-8f78-42daaecc20da None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:05.218521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024716a1-f2c5-4d3c-8f78-42daaecc20da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:05.260057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-024716a1-f2c5-4d3c-8f78-42daaecc20da None None] MFA Rules not processed for user `bd08361520db465bb9069326d03d0307`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:05.261566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-024716a1-f2c5-4d3c-8f78-42daaecc20da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:05.362764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1193/2378] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:05.578699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8649f-c1a0-4edb-9969-f6c5e592dcf8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:05.579026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8649f-c1a0-4edb-9969-f6c5e592dcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:05.579371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8649f-c1a0-4edb-9969-f6c5e592dcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:05.611204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0cf8649f-c1a0-4edb-9969-f6c5e592dcf8 None None] MFA Rules not processed for user `9102e895cbad460cb51e57416e705474`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:05.612511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0cf8649f-c1a0-4edb-9969-f6c5e592dcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:05.618717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:05.694676 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1186/2379] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:05.699626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024716a1-f2c5-4d3c-8f78-42daaecc20da None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:05.711067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024716a1-f2c5-4d3c-8f78-42daaecc20da None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:05.724017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:05.724768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:05.725046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:05.725320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:05.726131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:05.728277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:05.767595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1194/2380] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:05:05 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:05:05.768795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8649f-c1a0-4edb-9969-f6c5e592dcf8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:05.780398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8649f-c1a0-4edb-9969-f6c5e592dcf8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:05.836152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:05.836826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:05.837112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:05.837488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:05.838120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:05.840030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:05.862311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1187/2381] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:05:05 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:05:05.921663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:05.932761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98eeaa-8b24-4479-9785-ff07cf4d61c0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:05.937353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:05.937954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:05.938231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:05.938500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:05.939141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:05.940833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:05.952028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1195/2382] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:05.956620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:05.966407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9303b4-988d-4d0f-a57c-7f67c6d429e7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:05.971033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:05.971945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:05.973030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:05.973661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] PATH_INFO: `/v3/projects/140edcf0564b41ee9b3fbbae1fae253e/users/9f92db03b80c4bd18527e9b8edfa5ea3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:05.988565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=140edcf0564b41ee9b3fbbae1fae253e, user_id=9f92db03b80c4bd18527e9b8edfa5ea3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:05.991279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:06.006337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1188/2383] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:05:05 2022] PUT /identity/v3/projects/140edcf0564b41ee9b3fbbae1fae253e/users/9f92db03b80c4bd18527e9b8edfa5ea3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:06.011339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:06.021528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2274822-b70a-4d72-8a2c-48a7e002af24 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:06.075713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:06.082935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deeffe5-6c35-45ef-8a88-417c20c89636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:06.083742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deeffe5-6c35-45ef-8a88-417c20c89636 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:06.084103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deeffe5-6c35-45ef-8a88-417c20c89636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:06.084649 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deeffe5-6c35-45ef-8a88-417c20c89636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:06.085601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deeffe5-6c35-45ef-8a88-417c20c89636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:06.087262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deeffe5-6c35-45ef-8a88-417c20c89636 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:06.097700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1196/2384] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:06.101985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:06.112840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438957-caa0-45e4-bc26-659482d434fc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:06.117457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:06.118107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:06.118392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:06.118732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] PATH_INFO: `/v3/projects/140edcf0564b41ee9b3fbbae1fae253e/users/9f92db03b80c4bd18527e9b8edfa5ea3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:06.121239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140edcf0564b41ee9b3fbbae1fae253e, user_id=9f92db03b80c4bd18527e9b8edfa5ea3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:06.123876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:06.134555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1189/2385] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:05:06 2022] PUT /identity/v3/projects/140edcf0564b41ee9b3fbbae1fae253e/users/9f92db03b80c4bd18527e9b8edfa5ea3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:07.055892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:07.056486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:07.056638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:07.098859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] MFA Rules not processed for user `9f92db03b80c4bd18527e9b8edfa5ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:07.100109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:07.231002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1197/2386] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:07.252998 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:07.258375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:07.410801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd37b7b-78af-4979-bb82-2e132d8217b6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:07.415037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:07.415695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:07.416016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:07.416350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:07.430891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:07.480129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:07.481600 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:07.491566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1190/2387] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:05:08.436807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:08.439244 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:08.512589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf674278-a9f6-4d3d-8278-db3ce0e196c1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:08.517399 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:08.518017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:08.518294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:08.518569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] PATH_INFO: `/v3/users/1fafe6ce050545b8870f88d4ae0aa1ca` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:08.519682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fafe6ce050545b8870f88d4ae0aa1ca)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:08.521633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:08.567338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fafe6ce050545b8870f88d4ae0aa1ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.574021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fafe6ce050545b8870f88d4ae0aa1ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.580980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fafe6ce050545b8870f88d4ae0aa1ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.581582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fafe6ce050545b8870f88d4ae0aa1ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.592953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fafe6ce050545b8870f88d4ae0aa1ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.594326 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:08.600101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1198/2388] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:08 2022] DELETE /identity/v3/users/1fafe6ce050545b8870f88d4ae0aa1ca => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:08.769403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:08.782927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aae47-57c8-406b-a81a-b7ded47bd9d9 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:08.838916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:08.839537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:08.839813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:08.840087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] PATH_INFO: `/v3/projects/7c116c9cfbc144769d453e218b02739b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:08.841261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c116c9cfbc144769d453e218b02739b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:08.843155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:08.889364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c116c9cfbc144769d453e218b02739b', 'request_id': 'req-e0fd718d-7240-43df-bf36-933974339192'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.896616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c116c9cfbc144769d453e218b02739b', 'request_id': 'req-e0fd718d-7240-43df-bf36-933974339192'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.902892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c116c9cfbc144769d453e218b02739b', 'request_id': 'req-e0fd718d-7240-43df-bf36-933974339192'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.913644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] The token cache is being invalidate because project 7c116c9cfbc144769d453e218b02739b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:08.913955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.914583 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:08.916695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1191/2389] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:08 2022] DELETE /identity/v3/projects/7c116c9cfbc144769d453e218b02739b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:08.920704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:08.922094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:08.935193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3edbb-3058-4c12-a9ae-80209ba293ac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:09.007254 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:09.008388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:09.008734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:09.009017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] PATH_INFO: `/v3/users/753b62f8f2d24e66b5dba752b8797c3d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:09.010924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] RBAC: Authorizing `identity:delete_user(user_id=753b62f8f2d24e66b5dba752b8797c3d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:09.013255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:09.022203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:09.024520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:09.062879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753b62f8f2d24e66b5dba752b8797c3d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.071153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753b62f8f2d24e66b5dba752b8797c3d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.076463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:09.079944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753b62f8f2d24e66b5dba752b8797c3d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.080596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753b62f8f2d24e66b5dba752b8797c3d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.091090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd718d-7240-43df-bf36-933974339192 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:09.094066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753b62f8f2d24e66b5dba752b8797c3d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.094882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9f171-0836-48c9-8e9d-e8c22c3efa53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:09.095481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9f171-0836-48c9-8e9d-e8c22c3efa53 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:09.095756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9f171-0836-48c9-8e9d-e8c22c3efa53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:09.096023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9f171-0836-48c9-8e9d-e8c22c3efa53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:09.097802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-41b9f171-0836-48c9-8e9d-e8c22c3efa53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:09.100135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1199/2390] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:08 2022] DELETE /identity/v3/users/753b62f8f2d24e66b5dba752b8797c3d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:05:09.176534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9f171-0836-48c9-8e9d-e8c22c3efa53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:09.178020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9f171-0836-48c9-8e9d-e8c22c3efa53 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:09.192526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1192/2391] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:09.496618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:09.499011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:09.588529 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b36b4-27eb-4c34-8cac-14fd361e102c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:09.593785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:09.594773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:09.595045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:09.595318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] PATH_INFO: `/v3/projects/8cae06786c4845bab7927200f9dc48b2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:09.596763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cae06786c4845bab7927200f9dc48b2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:09.600163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:09.628633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:09.659186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cae06786c4845bab7927200f9dc48b2', 'request_id': 'req-4a118a0c-41b2-4315-aefb-a520cd008ac3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.665753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cae06786c4845bab7927200f9dc48b2', 'request_id': 'req-4a118a0c-41b2-4315-aefb-a520cd008ac3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.671621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cae06786c4845bab7927200f9dc48b2', 'request_id': 'req-4a118a0c-41b2-4315-aefb-a520cd008ac3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.690506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] The token cache is being invalidate because project 8cae06786c4845bab7927200f9dc48b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:09.690804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.691423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:09.693098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1200/2392] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:09 2022] DELETE /identity/v3/projects/8cae06786c4845bab7927200f9dc48b2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:05:16.622134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:16.622608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:16.622759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:16.648409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] MFA Rules not processed for user `68fcefa4a0db4bd6a051b1446e1a0d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:16.649607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:16.703364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1193/2393] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:19.687646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:20.729771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:20.731354 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:20.787910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a118a0c-41b2-4315-aefb-a520cd008ac3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:20.791563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:20.792158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:20.792464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:20.792736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:20.794448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:20.879614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:20.881430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:20.904004 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1201/2394] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:22.404818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:22.406281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:22.418495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8944c-472b-4373-a927-f8aed6f8129b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:22.422016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:22.422608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:22.422877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:22.423142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:22.424830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:22.437964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:22.439556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:22.451228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1194/2395] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:24.918022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:24.919449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:24.983917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f5a45-302c-4ca9-a9c4-653ece28fff6 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:24.988576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:24.989198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:24.989476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:24.989750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] PATH_INFO: `/v3/users/9102e895cbad460cb51e57416e705474` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:24.991338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9102e895cbad460cb51e57416e705474)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:24.995456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:25.014637 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:25.045654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9102e895cbad460cb51e57416e705474'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.052320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9102e895cbad460cb51e57416e705474'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.059866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9102e895cbad460cb51e57416e705474'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.060499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9102e895cbad460cb51e57416e705474'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.072406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9102e895cbad460cb51e57416e705474'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.078487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1202/2396] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:24 2022] DELETE /identity/v3/users/9102e895cbad460cb51e57416e705474 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:25.253754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:25.264869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59c3ac-e241-4fc7-8ae2-89f8cc8873af service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:25.323037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:25.323660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:25.323935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:25.324209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] PATH_INFO: `/v3/projects/70f27a79cbba4b0ba02139e1f5ee193c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:25.325342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f27a79cbba4b0ba02139e1f5ee193c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:25.327241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:25.357087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:25.373059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f27a79cbba4b0ba02139e1f5ee193c', 'request_id': 'req-000d36f1-8b65-430a-b268-d976b8f80fad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.380040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f27a79cbba4b0ba02139e1f5ee193c', 'request_id': 'req-000d36f1-8b65-430a-b268-d976b8f80fad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.386303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f27a79cbba4b0ba02139e1f5ee193c', 'request_id': 'req-000d36f1-8b65-430a-b268-d976b8f80fad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.397314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] The token cache is being invalidate because project 70f27a79cbba4b0ba02139e1f5ee193c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:25.397608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.398264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.399756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1195/2397] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:25 2022] DELETE /identity/v3/projects/70f27a79cbba4b0ba02139e1f5ee193c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:25.403813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:25.405261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:25.418431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c46c78-072d-4d79-953e-68127ccd1850 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:25.475654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:25.476310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:25.476593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:25.476866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] PATH_INFO: `/v3/users/9f92db03b80c4bd18527e9b8edfa5ea3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:25.478014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f92db03b80c4bd18527e9b8edfa5ea3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:25.479936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:25.523386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f92db03b80c4bd18527e9b8edfa5ea3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.530767 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f92db03b80c4bd18527e9b8edfa5ea3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.537225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f92db03b80c4bd18527e9b8edfa5ea3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.537856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f92db03b80c4bd18527e9b8edfa5ea3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.550132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b492dfb6-a8b3-4d11-902e-07d7758be91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f92db03b80c4bd18527e9b8edfa5ea3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.555722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1203/2398] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:25 2022] DELETE /identity/v3/users/9f92db03b80c4bd18527e9b8edfa5ea3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:25.728178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:25.729904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:25.743739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d36f1-8b65-430a-b268-d976b8f80fad admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:25.820612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:25.821919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:25.822297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:25.822581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] PATH_INFO: `/v3/projects/140edcf0564b41ee9b3fbbae1fae253e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:25.824067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=140edcf0564b41ee9b3fbbae1fae253e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:25.826163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:25.877883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140edcf0564b41ee9b3fbbae1fae253e', 'request_id': 'req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.886820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140edcf0564b41ee9b3fbbae1fae253e', 'request_id': 'req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.896590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140edcf0564b41ee9b3fbbae1fae253e', 'request_id': 'req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.918637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] The token cache is being invalidate because project 140edcf0564b41ee9b3fbbae1fae253e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:25.919012 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.919871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ae88d0a-48bf-406b-b4b2-f21538dd69ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:25.921707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1196/2399] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:25 2022] DELETE /identity/v3/projects/140edcf0564b41ee9b3fbbae1fae253e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:05:25.926468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f7b8-171d-4505-861b-01965977973f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:25.926980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f7b8-171d-4505-861b-01965977973f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:25.927107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f7b8-171d-4505-861b-01965977973f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:25.948501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4cd6f7b8-171d-4505-861b-01965977973f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:25.949708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4cd6f7b8-171d-4505-861b-01965977973f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:25.958727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:26.011467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1204/2400] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:26.021501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2311e7-cfbe-4ed1-a686-8e04e96edc4a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.021792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2311e7-cfbe-4ed1-a686-8e04e96edc4a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.022096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2311e7-cfbe-4ed1-a686-8e04e96edc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.041717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fe2311e7-cfbe-4ed1-a686-8e04e96edc4a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:26.042927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fe2311e7-cfbe-4ed1-a686-8e04e96edc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:26.050842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1197/2401] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:26.055240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f7b8-171d-4505-861b-01965977973f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.065993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f7b8-171d-4505-861b-01965977973f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.070581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.071172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.072074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.072402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.073048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:05:26.073348 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.074973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.086388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1205/2402] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:05:26.090841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2311e7-cfbe-4ed1-a686-8e04e96edc4a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.099952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2311e7-cfbe-4ed1-a686-8e04e96edc4a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.104627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.105217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.105490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.105758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.106392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.108229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.144850 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1198/2403] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:05:26 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:05:26.149062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.159406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de66d8-1001-4648-b7bf-ded11e1941b1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.214684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.215285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.215559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.215830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.216489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.218416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.240491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1206/2404] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:05:26 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:05:26.244598 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.254858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d8194-7471-43d7-9c2d-a9c9c8fa7200 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.259917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.260681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.260960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.261230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.261872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.263462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.265716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:26.277322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1199/2405] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:26.281122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.292317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58de43b-bfd6-42c3-9470-d9c144aad458 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.298959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9bead-d661-41d5-a82e-801d3fc37c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.299421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9bead-d661-41d5-a82e-801d3fc37c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.299696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9bead-d661-41d5-a82e-801d3fc37c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.299969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9bead-d661-41d5-a82e-801d3fc37c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.300656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9bead-d661-41d5-a82e-801d3fc37c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.302272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9bead-d661-41d5-a82e-801d3fc37c5e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.314575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1207/2406] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:26.319157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.330985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea68dce-c2f6-4ee6-ae38-e9cd4232f91d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.336777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.338023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.338296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.338568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] PATH_INFO: `/v3/projects/534c547142ae411e80ad3156d496f874/users/fd4c25630f4b4f00865000a0ae99b96d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.357525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=534c547142ae411e80ad3156d496f874, user_id=fd4c25630f4b4f00865000a0ae99b96d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.360417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.375696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1200/2407] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:05:26 2022] PUT /identity/v3/projects/534c547142ae411e80ad3156d496f874/users/fd4c25630f4b4f00865000a0ae99b96d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:05:26.388367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6aabe-c5b9-4f94-b57c-328b221f805e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.388741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6aabe-c5b9-4f94-b57c-328b221f805e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.389073 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6aabe-c5b9-4f94-b57c-328b221f805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.420953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-78c6aabe-c5b9-4f94-b57c-328b221f805e None None] MFA Rules not processed for user `fd4c25630f4b4f00865000a0ae99b96d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:26.422479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-78c6aabe-c5b9-4f94-b57c-328b221f805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:26.498505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1208/2408] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:26.536775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.538488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:26.594502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7775-1ae5-4b2a-ad01-46900b42e0fd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.598075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c5cc9-059e-44a8-aac3-a1d2df643e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.598689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5cc9-059e-44a8-aac3-a1d2df643e90 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.599125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5cc9-059e-44a8-aac3-a1d2df643e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.599558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5cc9-059e-44a8-aac3-a1d2df643e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.607720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c5cc9-059e-44a8-aac3-a1d2df643e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.609235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c5cc9-059e-44a8-aac3-a1d2df643e90 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.618033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1201/2409] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:26.633524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6aabe-c5b9-4f94-b57c-328b221f805e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.644741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:26.698461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6aabe-c5b9-4f94-b57c-328b221f805e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.703226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.703844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.704117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.704425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] PATH_INFO: `/v3/users/fd4c25630f4b4f00865000a0ae99b96d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.705532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4c25630f4b4f00865000a0ae99b96d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.707481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.752003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4c25630f4b4f00865000a0ae99b96d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:26.758645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4c25630f4b4f00865000a0ae99b96d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:26.766091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4c25630f4b4f00865000a0ae99b96d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:26.766751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4c25630f4b4f00865000a0ae99b96d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:26.778352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4c25630f4b4f00865000a0ae99b96d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:26.784072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1209/2410] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:26 2022] DELETE /identity/v3/users/fd4c25630f4b4f00865000a0ae99b96d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:26.788606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.788899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.789177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.808353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:26.809560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:26.865772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1202/2411] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:26.875864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:26.877282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:26.934162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75caa848-c7a4-4186-94bb-6846fc026802 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:26.937889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4681407-f36e-4401-ac57-a0770de71086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:26.938522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4681407-f36e-4401-ac57-a0770de71086 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:26.938790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4681407-f36e-4401-ac57-a0770de71086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:26.939067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4681407-f36e-4401-ac57-a0770de71086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:26.950541 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4681407-f36e-4401-ac57-a0770de71086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:26.952027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4681407-f36e-4401-ac57-a0770de71086 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:26.961410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1210/2412] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:27.249663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:27.251089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:27.267667 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b65074-6440-4a93-b6c4-d77241daa5df None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:27.272512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:27.273290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.273588 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.273867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] PATH_INFO: `/v3/projects/534c547142ae411e80ad3156d496f874` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.275041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=534c547142ae411e80ad3156d496f874)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:27.276901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:27.330003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '534c547142ae411e80ad3156d496f874', 'request_id': 'req-44187b62-a280-4e97-83ef-430a81b5fa60'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:27.337362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '534c547142ae411e80ad3156d496f874', 'request_id': 'req-44187b62-a280-4e97-83ef-430a81b5fa60'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:27.342821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '534c547142ae411e80ad3156d496f874', 'request_id': 'req-44187b62-a280-4e97-83ef-430a81b5fa60'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:27.350286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:27.354614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] The token cache is being invalidate because project 534c547142ae411e80ad3156d496f874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:27.355114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:27.356192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44187b62-a280-4e97-83ef-430a81b5fa60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:27.357861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1203/2413] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:27 2022] DELETE /identity/v3/projects/534c547142ae411e80ad3156d496f874 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:05:27.362879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90e6d-507e-4837-9a3d-a9ac16cb9390 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.363173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90e6d-507e-4837-9a3d-a9ac16cb9390 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.363453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90e6d-507e-4837-9a3d-a9ac16cb9390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.383423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-96f90e6d-507e-4837-9a3d-a9ac16cb9390 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:27.384952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-96f90e6d-507e-4837-9a3d-a9ac16cb9390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:27.412768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:27.456100 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1211/2414] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:27.464040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2059e-4f37-4309-9578-e2f8f1fcb946 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.464358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2059e-4f37-4309-9578-e2f8f1fcb946 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.464635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2059e-4f37-4309-9578-e2f8f1fcb946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.489359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-09d2059e-4f37-4309-9578-e2f8f1fcb946 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:27.491881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-09d2059e-4f37-4309-9578-e2f8f1fcb946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:27.504696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1204/2415] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:05:27.509009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f90e6d-507e-4837-9a3d-a9ac16cb9390 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:27.522276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f90e6d-507e-4837-9a3d-a9ac16cb9390 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:27.532306 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:27.533016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.533429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.533570 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.534219 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:05:27.534526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:27.536277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:27.553487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1212/2416] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:05:27.554557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2059e-4f37-4309-9578-e2f8f1fcb946 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:27.573669 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2059e-4f37-4309-9578-e2f8f1fcb946 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:27.578703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:27.579794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.580113 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.580649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.581291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:27.583245 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:27.626667 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1205/2417] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:05:27 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:05:27.631002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:27.645420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfab53-c37f-41e5-9abc-7c8be2437fe7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:27.724784 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:27.725378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.725655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.726310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.726622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:27.729196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:27.755754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1213/2418] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:05:27 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:05:27.760140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:27.774310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fe8e9-83cf-4d1e-ae74-84f33f03fd0e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:27.800504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:27.801634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.801920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.802200 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.802855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:27.804575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:27.820426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1206/2419] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:27.822560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:27.833343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426db77d-306e-4476-b2f6-cc8f026c60f1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:27.839867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea12ac4-9080-4243-9c28-f258b58246e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:27.841159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea12ac4-9080-4243-9c28-f258b58246e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.841681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea12ac4-9080-4243-9c28-f258b58246e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.842761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea12ac4-9080-4243-9c28-f258b58246e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.843662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea12ac4-9080-4243-9c28-f258b58246e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:27.845409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea12ac4-9080-4243-9c28-f258b58246e4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:27.861248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1214/2420] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:05:27.862277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:27.880497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d2ebf-eab1-4b31-9fbb-742cee3cb7bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:27.894950 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:27.896297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.896696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.896851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] PATH_INFO: `/v3/projects/132faa65e64a4205922d82d962fc62d5/users/e58a9e6d9e4c437085f36d3135c16467/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.923096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132faa65e64a4205922d82d962fc62d5, user_id=e58a9e6d9e4c437085f36d3135c16467, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:27.926909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:27.948279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1207/2421] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:05:27 2022] PUT /identity/v3/projects/132faa65e64a4205922d82d962fc62d5/users/e58a9e6d9e4c437085f36d3135c16467/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:05:27.959915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0f20a-b538-487d-8a47-f8513d05bdc4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:27.960470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0f20a-b538-487d-8a47-f8513d05bdc4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:27.960470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0f20a-b538-487d-8a47-f8513d05bdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:27.991634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-28e0f20a-b538-487d-8a47-f8513d05bdc4 None None] MFA Rules not processed for user `e58a9e6d9e4c437085f36d3135c16467`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:27.992910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-28e0f20a-b538-487d-8a47-f8513d05bdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:28.096833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1215/2422] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:28.107931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:28.110316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:28.194593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99590f92-d412-40e5-966f-b12902aa78a3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:28.199077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:28.199685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:28.199960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:28.200233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:28.211362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:28.212877 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:28.228404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1208/2423] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:29.644382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701049d-59b0-4a3e-a681-b88baf23fb88 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:29.644382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701049d-59b0-4a3e-a681-b88baf23fb88 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:29.644382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701049d-59b0-4a3e-a681-b88baf23fb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:29.678864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c701049d-59b0-4a3e-a681-b88baf23fb88 None None] MFA Rules not processed for user `e58a9e6d9e4c437085f36d3135c16467`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:29.680009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c701049d-59b0-4a3e-a681-b88baf23fb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:29.688081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1216/2424] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:38.219508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:39.660119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:48.225285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:48.226833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:48.299290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b9826-bf59-4c66-80d9-16a37fd36e7d service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:48.304705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:48.305302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:48.306133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:48.306408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:48.308099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:48.387219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:48.388826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:48.406971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1209/2425] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:05:48.502924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a431e9-dda2-4009-b37c-719a7f8543d0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:48.503393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a431e9-dda2-4009-b37c-719a7f8543d0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:48.503598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a431e9-dda2-4009-b37c-719a7f8543d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:48.562725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-99a431e9-dda2-4009-b37c-719a7f8543d0 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:48.564065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-99a431e9-dda2-4009-b37c-719a7f8543d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:48.630333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1217/2426] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 02:05:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:48.676231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:48.677682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:48.748539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:48.771836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3ef45-b84b-4f7f-a466-eb8a1fe3f713 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:48.783309 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:48.784207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:48.784544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:48.784986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:48.794398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:48.796027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:48.803985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1210/2427] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:05:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:05:49.290074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88226-8b36-43f5-ac93-7cd14bf503a0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:49.290385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88226-8b36-43f5-ac93-7cd14bf503a0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:49.290657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88226-8b36-43f5-ac93-7cd14bf503a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:49.310030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-53b88226-8b36-43f5-ac93-7cd14bf503a0 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:49.311207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-53b88226-8b36-43f5-ac93-7cd14bf503a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:49.318238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1218/2428] 10.222.0.50 () {58 vars in 967 bytes} [Tue Aug 30 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:49.331263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:49.341766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ad31e-7d67-4daa-ba52-1f469ee8e7f5 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:49.346811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:49.347400 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:49.347682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:49.347957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:49.356224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:49.357867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:49.365829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1211/2429] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:05:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:05:49.574622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317e43d-9a0c-4691-a246-9e9da3ac3df4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:49.575118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317e43d-9a0c-4691-a246-9e9da3ac3df4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:49.575260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317e43d-9a0c-4691-a246-9e9da3ac3df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:49.593726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4317e43d-9a0c-4691-a246-9e9da3ac3df4 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:49.594899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4317e43d-9a0c-4691-a246-9e9da3ac3df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:49.601964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1219/2430] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:49.614493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:49.625718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cc821-7c71-44cf-a55f-df86bcfacac6 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:49.630746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:49.631336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:49.631619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:49.631893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:49.640054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:49.641657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:49.649623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1212/2431] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:05:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:05:49.973828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96547b-1a12-4e59-9a6e-b0e72afec33f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:49.974117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96547b-1a12-4e59-9a6e-b0e72afec33f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:49.974393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96547b-1a12-4e59-9a6e-b0e72afec33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:49.994141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2b96547b-1a12-4e59-9a6e-b0e72afec33f None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:49.995338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2b96547b-1a12-4e59-9a6e-b0e72afec33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:50.002496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1220/2432] 10.222.0.50 () {58 vars in 967 bytes} [Tue Aug 30 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:05:50.014945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:50.025876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ff251-8252-4dd1-acee-38dc75e1cb4d service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:50.030942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:50.031538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:50.031816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:50.032091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:50.040749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:50.042313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:50.050278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1213/2433] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:05:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:05:51.945940 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96547b-1a12-4e59-9a6e-b0e72afec33f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:51.958830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96547b-1a12-4e59-9a6e-b0e72afec33f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:51.964338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43e300-c92f-4065-b0fe-6d84e5328e64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:51.964952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43e300-c92f-4065-b0fe-6d84e5328e64 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:51.965237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43e300-c92f-4065-b0fe-6d84e5328e64 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:51.965516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43e300-c92f-4065-b0fe-6d84e5328e64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:51.967245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be43e300-c92f-4065-b0fe-6d84e5328e64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:52.047559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43e300-c92f-4065-b0fe-6d84e5328e64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:52.049166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43e300-c92f-4065-b0fe-6d84e5328e64 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:52.067220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1221/2434] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:52.147475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:52.211843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65af99-e2d6-45a6-a776-bcecd148708a service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:52.216430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:52.217085 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:52.217362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:52.217637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] PATH_INFO: `/v3/users/e58a9e6d9e4c437085f36d3135c16467` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:52.218752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58a9e6d9e4c437085f36d3135c16467)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:52.220710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:52.270249 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58a9e6d9e4c437085f36d3135c16467'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.282684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58a9e6d9e4c437085f36d3135c16467'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.291033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58a9e6d9e4c437085f36d3135c16467'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.291954 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58a9e6d9e4c437085f36d3135c16467'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.292646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58a9e6d9e4c437085f36d3135c16467'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.307900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1214/2435] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:05:52 2022] DELETE /identity/v3/users/e58a9e6d9e4c437085f36d3135c16467 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:05:52.312889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:52.313288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:52.313590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:52.345111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:05:52.346475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:52.355592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:52.415268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1222/2436] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:05:52.425358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:52.426796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:52.448545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:05:52.485629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b10457-084b-48b9-b30c-5e2bcdd4c8f9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:52.489366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e12e40-43e0-42a8-b9a0-6cab6c3af759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:52.489964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e12e40-43e0-42a8-b9a0-6cab6c3af759 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:52.490240 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e12e40-43e0-42a8-b9a0-6cab6c3af759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:52.490508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e12e40-43e0-42a8-b9a0-6cab6c3af759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:52.499947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e12e40-43e0-42a8-b9a0-6cab6c3af759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:52.501390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e12e40-43e0-42a8-b9a0-6cab6c3af759 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:52.511648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1215/2437] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:05:52.678362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:05:52.679869 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:05:52.692509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66bdbe-3b92-4d72-9217-ee798fd2e29b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:05:52.697406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:05:52.698042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:05:52.698321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:05:52.698601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] PATH_INFO: `/v3/projects/132faa65e64a4205922d82d962fc62d5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:05:52.700383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=132faa65e64a4205922d82d962fc62d5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:05:52.702830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:05:52.749525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132faa65e64a4205922d82d962fc62d5', 'request_id': 'req-06b63f4a-7370-4e4e-a894-535588f81c6a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.755494 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132faa65e64a4205922d82d962fc62d5', 'request_id': 'req-06b63f4a-7370-4e4e-a894-535588f81c6a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.761313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132faa65e64a4205922d82d962fc62d5', 'request_id': 'req-06b63f4a-7370-4e4e-a894-535588f81c6a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.773951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] The token cache is being invalidate because project 132faa65e64a4205922d82d962fc62d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:05:52.774236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.774861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:05:52.776349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1223/2438] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:05:52 2022] DELETE /identity/v3/projects/132faa65e64a4205922d82d962fc62d5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:06:02.506565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=18 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:06:02.771035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:06:10.659487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:10.660027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:10.660139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:10.687876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] MFA Rules not processed for user `a06de37941264b979a3bd5aaf84ec73c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:06:10.689105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:10.742490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1216/2439] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:06:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:06:11.043721 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:11.045284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:11.101575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b63f4a-7370-4e4e-a894-535588f81c6a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:11.105140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:11.105763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:11.106038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:11.106308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:11.108018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:11.117280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:11.119233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:11.126993 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1224/2440] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:06:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:06:11.450571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:11.452029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:11.522230 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca0808-7dee-4c78-9699-ef7046ba1775 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:11.527095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f7b89-8285-4056-baad-a70e2d7417c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:11.527687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f7b89-8285-4056-baad-a70e2d7417c4 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:11.527963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f7b89-8285-4056-baad-a70e2d7417c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:11.528231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f7b89-8285-4056-baad-a70e2d7417c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:11.529949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f36f7b89-8285-4056-baad-a70e2d7417c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:11.541746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f7b89-8285-4056-baad-a70e2d7417c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:11.543330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f7b89-8285-4056-baad-a70e2d7417c4 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:11.553237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1217/2441] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:06:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:06:13.822840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:13.824260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:13.834598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce2ee8-2a23-4285-9e68-614b2bf2c79e service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:13.838248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:13.838840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:13.839181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:13.839465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:13.841132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:13.850084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:13.851486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:13.859660 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1225/2442] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:06:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:06:14.865470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b42ed8-079e-42b6-995e-6d52e8433bfa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:14.866001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b42ed8-079e-42b6-995e-6d52e8433bfa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:14.866001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b42ed8-079e-42b6-995e-6d52e8433bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:14.884865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-52b42ed8-079e-42b6-995e-6d52e8433bfa None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:06:14.886594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-52b42ed8-079e-42b6-995e-6d52e8433bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:14.947866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1218/2443] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 02:06:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:06:14.960840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:14.962840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:15.038657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86290834-5eb6-422b-a24a-2b77abd73a6d service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:15.044828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818af9f-2204-479a-8e4b-82c5d226dff8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:15.045438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818af9f-2204-479a-8e4b-82c5d226dff8 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:15.045713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818af9f-2204-479a-8e4b-82c5d226dff8 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:15.045984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818af9f-2204-479a-8e4b-82c5d226dff8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:15.056941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818af9f-2204-479a-8e4b-82c5d226dff8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:15.058535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818af9f-2204-479a-8e4b-82c5d226dff8 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:15.067173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1226/2444] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:06:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:06:24.939059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:06:25.063632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=64 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:06:28.798120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45fffd-2e16-471c-8d4c-94e509940fef None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:28.798658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45fffd-2e16-471c-8d4c-94e509940fef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:28.798747 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45fffd-2e16-471c-8d4c-94e509940fef None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:28.800253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1219/2445] 10.222.0.50 () {54 vars in 921 bytes} [Tue Aug 30 02:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 02:06:28.805245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae0966-10c1-4506-835c-5e478ce9f5c1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:28.805682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae0966-10c1-4506-835c-5e478ce9f5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:28.805804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae0966-10c1-4506-835c-5e478ce9f5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:28.844496 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8eae0966-10c1-4506-835c-5e478ce9f5c1 None None] MFA Rules not processed for user `95b365b4794c4edebe306e09336c42f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:06:28.845975 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8eae0966-10c1-4506-835c-5e478ce9f5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:28.912644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1227/2446] 10.222.0.50 () {58 vars in 1018 bytes} [Tue Aug 30 02:06:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:06:28.924827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45fffd-2e16-471c-8d4c-94e509940fef None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:28.926535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-de45fffd-2e16-471c-8d4c-94e509940fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:29.004643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45fffd-2e16-471c-8d4c-94e509940fef None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:29.008780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88273fe4807b4f08911298b7bf0c3653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:29.009370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:29.009645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:29.009914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:29.019928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:29.021422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:29.031799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1220/2447] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:06:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:06:30.952785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:30.953260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:30.953366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:30.972709 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:06:30.973899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:30.981530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1228/2448] 10.222.0.50 () {58 vars in 1009 bytes} [Tue Aug 30 02:06:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:06:30.993843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:31.005025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9ed7e-dbbc-4730-b3a7-85e9aec2e0ad service glance] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:31.009985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29d092-fe77-42a6-9cc8-d7736186993a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:31.010840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29d092-fe77-42a6-9cc8-d7736186993a service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:31.011119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29d092-fe77-42a6-9cc8-d7736186993a service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:31.011392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29d092-fe77-42a6-9cc8-d7736186993a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:31.019184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29d092-fe77-42a6-9cc8-d7736186993a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:31.020780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29d092-fe77-42a6-9cc8-d7736186993a service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:31.028736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1221/2449] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:06:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 02:06:40.966985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=40 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:06:41.025581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=41 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:06:52.261196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:52.262600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:52.280084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad93fc1-bdb3-4ed6-8ff3-7d16dc28794f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:52.285885 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:52.286595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:52.286970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:52.287334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:52.289132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:52.302707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:52.304311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:52.315205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1229/2450] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:06:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:06:55.027623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:55.028139 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:55.028284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:55.047939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:06:55.049153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:06:55.056641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1222/2451] 10.222.0.50 () {58 vars in 967 bytes} [Tue Aug 30 02:06:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:06:55.079416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:06:55.090466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d0c19-0922-49cf-a141-3a26ba6f959f service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:06:55.096022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458de03c-4e4b-4bf7-91b0-3ad4bdcdda06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:06:55.096645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458de03c-4e4b-4bf7-91b0-3ad4bdcdda06 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:06:55.096923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458de03c-4e4b-4bf7-91b0-3ad4bdcdda06 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:06:55.097196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458de03c-4e4b-4bf7-91b0-3ad4bdcdda06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:06:55.105218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458de03c-4e4b-4bf7-91b0-3ad4bdcdda06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:06:55.106775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458de03c-4e4b-4bf7-91b0-3ad4bdcdda06 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:06:55.115762 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1230/2452] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 02:07:05.042599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:05.112342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=12 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:11.233969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:11.235440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:11.256546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef718a-d371-4c73-999b-0bb3cd21b9ff None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:11.262577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62994477-085a-485c-a36a-e0c336300184 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:11.263463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62994477-085a-485c-a36a-e0c336300184 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:11.263941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62994477-085a-485c-a36a-e0c336300184 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:11.264457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62994477-085a-485c-a36a-e0c336300184 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:11.266826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-62994477-085a-485c-a36a-e0c336300184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:11.283414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62994477-085a-485c-a36a-e0c336300184 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:11.288319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62994477-085a-485c-a36a-e0c336300184 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:11.301026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1223/2453] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:07:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:07:12.899191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37111-b9f6-43a7-8995-25da1e89c429 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:12.899652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37111-b9f6-43a7-8995-25da1e89c429 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:12.899764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37111-b9f6-43a7-8995-25da1e89c429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:12.918991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f3b37111-b9f6-43a7-8995-25da1e89c429 None None] MFA Rules not processed for user `b5ef7dccfc5e43be993e201a19730a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:07:12.920155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f3b37111-b9f6-43a7-8995-25da1e89c429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:12.927876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1231/2454] 10.222.0.50 () {58 vars in 967 bytes} [Tue Aug 30 02:07:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:07:12.939807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62994477-085a-485c-a36a-e0c336300184 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:12.952695 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62994477-085a-485c-a36a-e0c336300184 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:12.959668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5956fdd8d6b54640970cde2705a83260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:12.960322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:12.960604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:12.960880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:12.969048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:12.971099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:12.978980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1224/2455] 10.222.0.50 () {60 vars in 1384 bytes} [Tue Aug 30 02:07:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 02:07:13.335881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b37111-b9f6-43a7-8995-25da1e89c429 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:13.348546 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b37111-b9f6-43a7-8995-25da1e89c429 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:13.354048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:13.354907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:13.355144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:13.355425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:13.357148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:13.438814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:13.441059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:13.459777 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1232/2456] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:07:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:07:19.284120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:19.287450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:19.361649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431bc51-2e8e-4646-9a80-b1524b974aa4 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:19.366195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:19.366823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:19.367218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:19.367515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] PATH_INFO: `/v3/users/6131a8d2e9374070bc770270c7a70519` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:19.368730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6131a8d2e9374070bc770270c7a70519)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:19.370597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:19.424838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6131a8d2e9374070bc770270c7a70519'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.440511 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6131a8d2e9374070bc770270c7a70519'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.448628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6131a8d2e9374070bc770270c7a70519'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.449417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6131a8d2e9374070bc770270c7a70519'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.450033 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6131a8d2e9374070bc770270c7a70519'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.464992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1225/2457] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:07:19 2022] DELETE /identity/v3/users/6131a8d2e9374070bc770270c7a70519 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:07:19.631813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:19.642555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170d5c4-5cfa-41a3-9a8b-6e5383e64901 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:19.698746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:19.699302 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:19.699572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:19.699844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] PATH_INFO: `/v3/projects/7351c123c59146cf80c23e0d1ffe80fb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:19.700996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=7351c123c59146cf80c23e0d1ffe80fb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:19.702914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:19.748356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7351c123c59146cf80c23e0d1ffe80fb', 'request_id': 'req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.754938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7351c123c59146cf80c23e0d1ffe80fb', 'request_id': 'req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.760424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7351c123c59146cf80c23e0d1ffe80fb', 'request_id': 'req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.763118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:19.775277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] The token cache is being invalidate because project 7351c123c59146cf80c23e0d1ffe80fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:07:19.775567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.776276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.777765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1233/2458] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:07:19 2022] DELETE /identity/v3/projects/7351c123c59146cf80c23e0d1ffe80fb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:07:19.781717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:19.783159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:19.796407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeee0a5-7899-4a47-a0a6-4a8d348da482 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:19.844129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:19.853944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:19.854565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:19.854841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:19.855112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] PATH_INFO: `/v3/users/5a80616de4534b7f87e5f0573d077a6e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:19.856213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a80616de4534b7f87e5f0573d077a6e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:19.858148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:19.901060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a80616de4534b7f87e5f0573d077a6e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.912906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a80616de4534b7f87e5f0573d077a6e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.919607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a80616de4534b7f87e5f0573d077a6e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.920281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a80616de4534b7f87e5f0573d077a6e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.920884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a80616de4534b7f87e5f0573d077a6e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:19.932743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1226/2459] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:07:19 2022] DELETE /identity/v3/users/5a80616de4534b7f87e5f0573d077a6e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:07:20.106249 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:20.107779 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:20.121645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612b643-388b-4bb4-8ae8-9a0c7ba9d6ca admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:20.177722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:20.178345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:20.178619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:20.178891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] PATH_INFO: `/v3/projects/7c5aad4db06146dbbd7036fff2667cd4` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:20.179986 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c5aad4db06146dbbd7036fff2667cd4)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:20.181953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:20.226274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c5aad4db06146dbbd7036fff2667cd4', 'request_id': 'req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:20.232198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c5aad4db06146dbbd7036fff2667cd4', 'request_id': 'req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:20.238112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c5aad4db06146dbbd7036fff2667cd4', 'request_id': 'req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:20.249627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] The token cache is being invalidate because project 7c5aad4db06146dbbd7036fff2667cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:07:20.249926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:20.250534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:20.252604 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1234/2460] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:07:20 2022] DELETE /identity/v3/projects/7c5aad4db06146dbbd7036fff2667cd4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:07:29.929291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:30.246629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:43.992129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:43.993558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:44.053892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66d806-0331-43ab-870e-c2fdc6e1c897 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:44.100555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:44.101090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:44.101372 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:44.101648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:44.103396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:44.159509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:44.160966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:44.170306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1227/2461] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:07:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:07:45.127391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:45.128808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:45.186547 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed17e2f-2f19-4f0e-8655-ab8e51e5b607 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:45.190127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:45.190718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:45.190992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:45.191260 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:45.192953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:45.252462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:45.253886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:45.263524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1235/2462] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:07:48.317754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:48.320067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:48.335729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9a2d2-99d2-46ef-b747-62cdb5b1470d service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:48.341780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:48.342427 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:48.342711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:48.343160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] PATH_INFO: `/v3/users/68fcefa4a0db4bd6a051b1446e1a0d2d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:48.344557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fcefa4a0db4bd6a051b1446e1a0d2d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:48.346504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:48.392984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fcefa4a0db4bd6a051b1446e1a0d2d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.404584 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fcefa4a0db4bd6a051b1446e1a0d2d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.405752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:48.413253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fcefa4a0db4bd6a051b1446e1a0d2d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.413923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fcefa4a0db4bd6a051b1446e1a0d2d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.414536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d44b296-485f-4d3b-8403-0970f7588286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fcefa4a0db4bd6a051b1446e1a0d2d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.426740 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1228/2463] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:07:48 2022] DELETE /identity/v3/users/68fcefa4a0db4bd6a051b1446e1a0d2d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:07:48.607011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:48.617977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec17713-ca20-4aaf-9541-181a4c462670 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:48.675501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:48.676124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:48.676432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:48.676707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] PATH_INFO: `/v3/projects/f90f06a31df841b89ed6874d20340475` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:48.677840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90f06a31df841b89ed6874d20340475)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:48.679767 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:48.725171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90f06a31df841b89ed6874d20340475', 'request_id': 'req-7f33191f-0f36-41e8-889f-2142e88fed14'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.727029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:48.731732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90f06a31df841b89ed6874d20340475', 'request_id': 'req-7f33191f-0f36-41e8-889f-2142e88fed14'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.737563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90f06a31df841b89ed6874d20340475', 'request_id': 'req-7f33191f-0f36-41e8-889f-2142e88fed14'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.748915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] The token cache is being invalidate because project f90f06a31df841b89ed6874d20340475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:07:48.749196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.749830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f33191f-0f36-41e8-889f-2142e88fed14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:07:48.751364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1236/2464] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:07:48 2022] DELETE /identity/v3/projects/f90f06a31df841b89ed6874d20340475 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:07:48.756340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9549d39-3ab2-4e0e-a776-62e1adfbc4af None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:48.756650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9549d39-3ab2-4e0e-a776-62e1adfbc4af None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:48.756925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9549d39-3ab2-4e0e-a776-62e1adfbc4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:48.776817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f9549d39-3ab2-4e0e-a776-62e1adfbc4af None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:07:48.778014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f9549d39-3ab2-4e0e-a776-62e1adfbc4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:48.834648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1229/2465] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:07:48.842363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf56e1-2a30-44e7-8ba6-0ef80469431e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:48.842676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf56e1-2a30-44e7-8ba6-0ef80469431e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:48.842948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf56e1-2a30-44e7-8ba6-0ef80469431e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:48.862129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dcdf56e1-2a30-44e7-8ba6-0ef80469431e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:07:48.863301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dcdf56e1-2a30-44e7-8ba6-0ef80469431e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:48.871136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1237/2466] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:07:48.875199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9549d39-3ab2-4e0e-a776-62e1adfbc4af None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:48.885458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9549d39-3ab2-4e0e-a776-62e1adfbc4af None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:48.890305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:48.890898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:48.891178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:48.891452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:48.892093 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:07:48.892431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:48.894013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:48.906114 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1230/2467] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:07:48.910216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf56e1-2a30-44e7-8ba6-0ef80469431e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:48.919431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf56e1-2a30-44e7-8ba6-0ef80469431e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:48.929451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:48.930054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:48.930328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:48.930595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:48.931220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:48.933183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:48.961419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1238/2468] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:07:48 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:07:48.965532 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:48.975352 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22d47c-6d41-475f-872c-33e8aed1c55c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:49.030691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:49.031294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:49.031578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:49.031858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:49.032527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:49.034441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:49.054896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1231/2469] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:07:48 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:07:49.059010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:49.069693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3060f2-121a-4458-9701-8bbb8a157e60 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:49.074271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:49.074856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:49.075134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:49.075408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:49.076051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:49.077678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:49.087895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1239/2470] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:07:49.091569 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:49.101327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91a227-154a-4c65-8846-c517cb484628 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:49.105851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618174c-f251-4048-89aa-2dea5d780b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:49.106437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618174c-f251-4048-89aa-2dea5d780b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:49.106711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618174c-f251-4048-89aa-2dea5d780b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:49.106980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618174c-f251-4048-89aa-2dea5d780b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:49.107603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618174c-f251-4048-89aa-2dea5d780b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:49.109207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618174c-f251-4048-89aa-2dea5d780b5b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:49.119550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1232/2471] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:07:49.123499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:49.133577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c3fa-5320-4282-8f78-dad9eaeabf25 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:49.138446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:49.139087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:49.139364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:49.139654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] PATH_INFO: `/v3/projects/ba9c4dee0873439db79da71d88c9dc22/users/87a351ad44c24b5191eb9e6807d8810a/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:49.154628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c4dee0873439db79da71d88c9dc22, user_id=87a351ad44c24b5191eb9e6807d8810a, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:49.157342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:49.180067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1240/2472] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:07:49 2022] PUT /identity/v3/projects/ba9c4dee0873439db79da71d88c9dc22/users/87a351ad44c24b5191eb9e6807d8810a/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:07:49.194633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cab00-1767-40a0-b504-97d5791bd906 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:49.194913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cab00-1767-40a0-b504-97d5791bd906 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:49.195191 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cab00-1767-40a0-b504-97d5791bd906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:49.222500 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cf0cab00-1767-40a0-b504-97d5791bd906 None None] MFA Rules not processed for user `87a351ad44c24b5191eb9e6807d8810a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:07:49.223706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cf0cab00-1767-40a0-b504-97d5791bd906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:49.262444 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:07:49.289659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1233/2473] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:07:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:07:49.300628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:07:49.302367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:49.359559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d75b11-8c3d-4205-950e-1af241152a93 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:07:49.363170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:07:49.363768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:49.364051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:49.364362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:49.372631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:07:49.374128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:07:49.383336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1241/2474] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:07:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:07:56.299294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985603f-2419-4a72-8207-df52b4a4e5ae None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:07:56.299682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985603f-2419-4a72-8207-df52b4a4e5ae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:07:56.299843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985603f-2419-4a72-8207-df52b4a4e5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:07:56.320022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e985603f-2419-4a72-8207-df52b4a4e5ae None None] MFA Rules not processed for user `bd08361520db465bb9069326d03d0307`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:07:56.321232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e985603f-2419-4a72-8207-df52b4a4e5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:07:56.376131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1234/2475] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:07:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:07:59.383718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=58,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:02.032127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:02.092776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e146f2b-0b27-44f0-8f74-b3779b4604d9 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:02.097083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:02.097688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.097963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.098232 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] PATH_INFO: `/v3/users/87a351ad44c24b5191eb9e6807d8810a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.099328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a351ad44c24b5191eb9e6807d8810a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:02.101343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:02.146080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a351ad44c24b5191eb9e6807d8810a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.152763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a351ad44c24b5191eb9e6807d8810a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.159830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a351ad44c24b5191eb9e6807d8810a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.160476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a351ad44c24b5191eb9e6807d8810a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.172220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a351ad44c24b5191eb9e6807d8810a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.178781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1242/2476] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:02 2022] DELETE /identity/v3/users/87a351ad44c24b5191eb9e6807d8810a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:02.183273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.183759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.183859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.202597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:02.203769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:02.259084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1235/2477] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:02.269301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:02.270688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:02.327549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8d71a-67d7-4ad9-a84e-8aa1fcb3a5ee admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:02.331181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86e3f9-32a0-4e76-9345-27d369340de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:02.331776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e3f9-32a0-4e76-9345-27d369340de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.332053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e3f9-32a0-4e76-9345-27d369340de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.332360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e3f9-32a0-4e76-9345-27d369340de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.341313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86e3f9-32a0-4e76-9345-27d369340de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:02.342736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86e3f9-32a0-4e76-9345-27d369340de4 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:02.352598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1243/2478] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:02.611268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:02.612722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:02.625699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a55f0-53fc-40c5-a841-1c1fd3837f56 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:02.630625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:02.631382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.631732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.632184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] PATH_INFO: `/v3/projects/ba9c4dee0873439db79da71d88c9dc22` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.633417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9c4dee0873439db79da71d88c9dc22)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:02.635306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:02.681637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9c4dee0873439db79da71d88c9dc22', 'request_id': 'req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.688818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9c4dee0873439db79da71d88c9dc22', 'request_id': 'req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.694547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9c4dee0873439db79da71d88c9dc22', 'request_id': 'req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.705180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] The token cache is being invalidate because project ba9c4dee0873439db79da71d88c9dc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:02.705463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.706074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a791035-d3cf-43ee-9fae-0ecb6afeee7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:02.707523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1236/2479] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:02 2022] DELETE /identity/v3/projects/ba9c4dee0873439db79da71d88c9dc22 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:02.712843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2878-373d-41ed-9629-63b02938c9f3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.713131 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2878-373d-41ed-9629-63b02938c9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.713405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2878-373d-41ed-9629-63b02938c9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.733383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-998a2878-373d-41ed-9629-63b02938c9f3 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:02.734562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-998a2878-373d-41ed-9629-63b02938c9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:02.750772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:02.794899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1244/2480] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:02.802653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd9ae0-906f-43ce-a9f1-6fac11192c4b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.802943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd9ae0-906f-43ce-a9f1-6fac11192c4b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.803216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd9ae0-906f-43ce-a9f1-6fac11192c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.817219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:02.823145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0cfd9ae0-906f-43ce-a9f1-6fac11192c4b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:02.824957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0cfd9ae0-906f-43ce-a9f1-6fac11192c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:02.832708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1237/2481] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:02.837303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a2878-373d-41ed-9629-63b02938c9f3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:02.847089 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a2878-373d-41ed-9629-63b02938c9f3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:02.852220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:02.852845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.853144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.853420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.854112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:02.854384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:02.855963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:02.867032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1245/2482] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:08:02.871045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd9ae0-906f-43ce-a9f1-6fac11192c4b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:02.879999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd9ae0-906f-43ce-a9f1-6fac11192c4b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:02.884608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:02.885214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:02.885471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:02.885736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:02.886603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:02.888477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:02.916305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1238/2483] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:02 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:02.920142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:02.930494 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d0def-9d7d-4c18-8944-06f952940a81 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:02.999346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:02.999950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:03.000226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:03.000528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:03.001160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:03.003117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:03.032036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1246/2484] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:02 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:03.033573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:03.043870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1df73-caa2-4a8c-a089-333b6f55041e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:03.048458 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:03.049049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:03.049323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:03.049592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:03.050223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:03.051789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:03.061672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1239/2485] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:03.065671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:03.076125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f934f8-903b-4774-bde9-813b03be7f6e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:03.080795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a49fe-3887-4782-b2b7-77a70734a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:03.081387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a49fe-3887-4782-b2b7-77a70734a2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:03.081658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a49fe-3887-4782-b2b7-77a70734a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:03.081923 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a49fe-3887-4782-b2b7-77a70734a2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:03.082583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a49fe-3887-4782-b2b7-77a70734a2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:03.084148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a49fe-3887-4782-b2b7-77a70734a2ce admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:03.095115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1247/2486] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:03.099194 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:03.109049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd279641-d161-4013-96e6-56c57ad4426e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:03.113441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:03.114073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:03.114350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:03.114621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] PATH_INFO: `/v3/projects/490396974b9940b982458eb2d6b1c512/users/86878536aef6401fa409917b574b9726/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:03.129314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490396974b9940b982458eb2d6b1c512, user_id=86878536aef6401fa409917b574b9726, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:03.132702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:03.145882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1240/2487] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:03 2022] PUT /identity/v3/projects/490396974b9940b982458eb2d6b1c512/users/86878536aef6401fa409917b574b9726/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:03.150934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09a7f9-a042-4c2e-b9f8-958dfe8fc143 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:03.151224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09a7f9-a042-4c2e-b9f8-958dfe8fc143 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:03.151499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09a7f9-a042-4c2e-b9f8-958dfe8fc143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:03.170162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cc09a7f9-a042-4c2e-b9f8-958dfe8fc143 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:03.171387 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cc09a7f9-a042-4c2e-b9f8-958dfe8fc143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:03.228543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1248/2488] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:03.238315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:03.240150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:03.296323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71df80-879e-4549-b481-2cb6289ff1b6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:03.299915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:03.300566 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:03.300839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:03.301121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:03.311370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:03.312958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:03.323050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1241/2489] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:03.928011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488afe2-1040-4c17-a841-15b0626e001a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:03.928357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488afe2-1040-4c17-a841-15b0626e001a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:03.928630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488afe2-1040-4c17-a841-15b0626e001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:03.956999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e488afe2-1040-4c17-a841-15b0626e001a None None] MFA Rules not processed for user `86878536aef6401fa409917b574b9726`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:03.958629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e488afe2-1040-4c17-a841-15b0626e001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:03.978080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:04.027592 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1249/2490] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:03 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:04.032061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:04.042268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77105d3d-01d2-4ae4-adb6-b753fba69714 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:04.046780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:04.047373 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:04.047648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:04.047919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:04.048591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:04.050484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:04.077523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1242/2491] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:04 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:04.081765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488afe2-1040-4c17-a841-15b0626e001a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:04.093531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488afe2-1040-4c17-a841-15b0626e001a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:04.154960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:04.155524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:04.155831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:04.156106 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:04.156809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:04.158771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:04.182124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1250/2492] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:04 2022] POST /identity/v3/users => generated 309 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:04.186214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:04.196743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25350f-9dbb-4468-8a05-a20136b03c25 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:04.201408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:04.202009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:04.202290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:04.202571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:04.203211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:04.204904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:04.216160 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1243/2493] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:04.220416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:04.230883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60c73e-0d38-4862-890d-26f941c8ca25 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:04.236065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:04.236798 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:04.237082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:04.237359 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] PATH_INFO: `/v3/projects/259a84efb9214b23889d2b21e26d28a8/users/18053420066d4099959308abe719de65/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:04.252983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259a84efb9214b23889d2b21e26d28a8, user_id=18053420066d4099959308abe719de65, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:04.255646 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:04.269284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1251/2494] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:04 2022] PUT /identity/v3/projects/259a84efb9214b23889d2b21e26d28a8/users/18053420066d4099959308abe719de65/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:04.273756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:04.283871 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c0fbf-7df3-4e15-82aa-ede289e6f830 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:04.342431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cba83-09c5-405f-ace8-4d1b5e8affa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:04.343034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cba83-09c5-405f-ace8-4d1b5e8affa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:04.343318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cba83-09c5-405f-ace8-4d1b5e8affa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:04.343593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cba83-09c5-405f-ace8-4d1b5e8affa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:04.344269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cba83-09c5-405f-ace8-4d1b5e8affa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:04.345869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cba83-09c5-405f-ace8-4d1b5e8affa7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:04.355471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1244/2495] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:04.359736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:04.370067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba60ec-7547-47a5-87a3-f368f63001e9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:04.374944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:04.375589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:04.375866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:04.376137 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] PATH_INFO: `/v3/projects/259a84efb9214b23889d2b21e26d28a8/users/18053420066d4099959308abe719de65/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:04.378077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=259a84efb9214b23889d2b21e26d28a8, user_id=18053420066d4099959308abe719de65, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:04.380757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:04.391445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1252/2496] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:04 2022] PUT /identity/v3/projects/259a84efb9214b23889d2b21e26d28a8/users/18053420066d4099959308abe719de65/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:05.018748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e38fa-8ed6-4f6e-b2d0-76d64c0a61de None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:05.019032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e38fa-8ed6-4f6e-b2d0-76d64c0a61de None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:05.019296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e38fa-8ed6-4f6e-b2d0-76d64c0a61de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:05.039364 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:05.050492 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bb2e38fa-8ed6-4f6e-b2d0-76d64c0a61de None None] MFA Rules not processed for user `18053420066d4099959308abe719de65`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:05.051711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb2e38fa-8ed6-4f6e-b2d0-76d64c0a61de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:05.127969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1245/2497] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:05.618730 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:05.620454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:05.699764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb33f10-267a-4194-9768-0dbaebc20f6c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:05.704801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:05.705434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:05.705718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:05.705991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:05.707717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:05.719506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:05.721118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:05.726996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:05.732401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1253/2498] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:06.189856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e38fa-8ed6-4f6e-b2d0-76d64c0a61de None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:06.251546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e38fa-8ed6-4f6e-b2d0-76d64c0a61de None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:06.256078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:06.256736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:06.257015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:06.257285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] PATH_INFO: `/v3/users/86878536aef6401fa409917b574b9726` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:06.258395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86878536aef6401fa409917b574b9726)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:06.260318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:06.308659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86878536aef6401fa409917b574b9726'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.320393 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86878536aef6401fa409917b574b9726'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.327163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86878536aef6401fa409917b574b9726'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.327861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86878536aef6401fa409917b574b9726'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.328526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86878536aef6401fa409917b574b9726'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.341538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1246/2499] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:06 2022] DELETE /identity/v3/users/86878536aef6401fa409917b574b9726 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:06.508308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:06.520865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502e26e-08ef-41e0-8f8c-bffa18a8a659 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:06.599752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:06.600443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:06.600728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:06.601006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] PATH_INFO: `/v3/projects/490396974b9940b982458eb2d6b1c512` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:06.602390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490396974b9940b982458eb2d6b1c512)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:06.604212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:06.661396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490396974b9940b982458eb2d6b1c512', 'request_id': 'req-73248735-9ee8-4673-a147-736edfbd38c6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.670229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490396974b9940b982458eb2d6b1c512', 'request_id': 'req-73248735-9ee8-4673-a147-736edfbd38c6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.676848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490396974b9940b982458eb2d6b1c512', 'request_id': 'req-73248735-9ee8-4673-a147-736edfbd38c6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.693023 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] The token cache is being invalidate because project 490396974b9940b982458eb2d6b1c512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:06.694092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.694840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.696580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1254/2500] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:06 2022] DELETE /identity/v3/projects/490396974b9940b982458eb2d6b1c512 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:06.700952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:06.702720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:06.716830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090f12b-01bc-4cc9-8064-1ad889150a09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:06.726801 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:06.781949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:06.782808 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:06.783092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:06.783369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] PATH_INFO: `/v3/users/18053420066d4099959308abe719de65` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:06.784535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=18053420066d4099959308abe719de65)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:06.786452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:06.851706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18053420066d4099959308abe719de65'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.869117 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18053420066d4099959308abe719de65'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.875897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18053420066d4099959308abe719de65'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.877213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18053420066d4099959308abe719de65'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.877342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e6c6a83c-5ec3-4edc-9ef6-e39c33757caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18053420066d4099959308abe719de65'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:06.891193 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1247/2501] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:06 2022] DELETE /identity/v3/users/18053420066d4099959308abe719de65 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:07.060371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.061938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:07.075676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73248735-9ee8-4673-a147-736edfbd38c6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.130804 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.131976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.132267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.132626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] PATH_INFO: `/v3/projects/259a84efb9214b23889d2b21e26d28a8` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.133745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259a84efb9214b23889d2b21e26d28a8)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.135665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.168814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:07.183205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259a84efb9214b23889d2b21e26d28a8', 'request_id': 'req-07a10fe9-4db2-4b0a-909e-9ac121688632'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:07.189393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259a84efb9214b23889d2b21e26d28a8', 'request_id': 'req-07a10fe9-4db2-4b0a-909e-9ac121688632'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:07.195382 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259a84efb9214b23889d2b21e26d28a8', 'request_id': 'req-07a10fe9-4db2-4b0a-909e-9ac121688632'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:07.206866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] The token cache is being invalidate because project 259a84efb9214b23889d2b21e26d28a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:07.207151 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:07.207772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-07a10fe9-4db2-4b0a-909e-9ac121688632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:07.209713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1255/2502] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:07 2022] DELETE /identity/v3/projects/259a84efb9214b23889d2b21e26d28a8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:07.214834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dd250-1760-4395-b091-c49588049df6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.215733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dd250-1760-4395-b091-c49588049df6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.215733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dd250-1760-4395-b091-c49588049df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.236636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-660dd250-1760-4395-b091-c49588049df6 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:07.237873 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-660dd250-1760-4395-b091-c49588049df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:07.296736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1248/2503] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:07.305144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091f9d-613b-488c-b206-1f22472b6c7e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.305526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091f9d-613b-488c-b206-1f22472b6c7e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.305826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091f9d-613b-488c-b206-1f22472b6c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.325290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3a091f9d-613b-488c-b206-1f22472b6c7e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:07.326470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3a091f9d-613b-488c-b206-1f22472b6c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:07.334275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1256/2504] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:07.338459 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dd250-1760-4395-b091-c49588049df6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.348202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dd250-1760-4395-b091-c49588049df6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.353349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.353938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.354217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.354493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.355165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:07.355466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.357087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.369013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1249/2505] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:08:07.373161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091f9d-613b-488c-b206-1f22472b6c7e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.382688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091f9d-613b-488c-b206-1f22472b6c7e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.387610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.388199 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.388504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.388776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.389398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.391283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.418976 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1257/2506] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:07 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:07.423244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.433349 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4d5ab-1f08-46d2-b816-46a2f72a173c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.450558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:07.492167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.492805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.493116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.493530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.494189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.496111 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.518420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1250/2507] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:07 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:07.523218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.533676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a367d-128f-4121-af32-6446ce7b1291 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.538491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.539088 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.539366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.539636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.540307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.541913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.551936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1258/2508] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:07.556162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.566089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc010b-9139-43bc-99bc-cb9c94b92fe6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.571679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a207fa9-3892-4b48-8233-b5fc891581e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.572320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a207fa9-3892-4b48-8233-b5fc891581e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.572603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a207fa9-3892-4b48-8233-b5fc891581e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.572879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a207fa9-3892-4b48-8233-b5fc891581e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.573543 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a207fa9-3892-4b48-8233-b5fc891581e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.575130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a207fa9-3892-4b48-8233-b5fc891581e8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.586420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1251/2509] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:07.590462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.600481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1edf-ec2b-406e-a4ff-b6a2efea1168 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.605086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.605729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.606008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.606283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] PATH_INFO: `/v3/projects/6cc8ee9f1d7842ebb755d2fd513fa859/users/bc8f3032a6a24b87abaa406bd3df6a6d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.620830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc8ee9f1d7842ebb755d2fd513fa859, user_id=bc8f3032a6a24b87abaa406bd3df6a6d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.623468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.637277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1259/2510] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:07 2022] PUT /identity/v3/projects/6cc8ee9f1d7842ebb755d2fd513fa859/users/bc8f3032a6a24b87abaa406bd3df6a6d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:07.647434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69823cd-3c45-499d-aea3-f86af01807e2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.647725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69823cd-3c45-499d-aea3-f86af01807e2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.648004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69823cd-3c45-499d-aea3-f86af01807e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.674997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d69823cd-3c45-499d-aea3-f86af01807e2 None None] MFA Rules not processed for user `bc8f3032a6a24b87abaa406bd3df6a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:07.676211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d69823cd-3c45-499d-aea3-f86af01807e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:07.744661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1252/2511] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:07.745831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.810673 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805fe76-257a-4de1-a6e9-2a781d99618d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.815568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.816188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.816528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.816797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.817420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.819967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.847161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1260/2512] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:07 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:07.850967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69823cd-3c45-499d-aea3-f86af01807e2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.861768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69823cd-3c45-499d-aea3-f86af01807e2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.935436 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.936050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.936363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.936644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.937272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.939226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.961421 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1253/2513] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:07 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:07.965877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:07.976914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8f23-dda1-4e3c-8012-1147a8c0a2aa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:07.981583 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:07.982178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:07.982452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:07.982850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:07.983486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:07.985098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:07.996768 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1261/2514] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:08.000856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:08.011602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e674551-8084-43fa-83b9-e5db5fefbb01 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:08.016340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:08.017029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.017310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.017586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] PATH_INFO: `/v3/projects/b1855c89803e4c48b6d75a16ce901722/users/a1752877cb4d4198ac4ae6d62956a00b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.034925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1855c89803e4c48b6d75a16ce901722, user_id=a1752877cb4d4198ac4ae6d62956a00b, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:08.038314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:08.040518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:08.052319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1254/2515] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:07 2022] PUT /identity/v3/projects/b1855c89803e4c48b6d75a16ce901722/users/a1752877cb4d4198ac4ae6d62956a00b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:08.057118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:08.068334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1a402-8af9-4ceb-9cd7-135103aa42f2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:08.100576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:08.132109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b93c8-c1f8-40ef-9753-05e63cdb3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:08.132760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b93c8-c1f8-40ef-9753-05e63cdb3be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.133134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b93c8-c1f8-40ef-9753-05e63cdb3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.133314 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b93c8-c1f8-40ef-9753-05e63cdb3be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.133956 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b93c8-c1f8-40ef-9753-05e63cdb3be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:08.135550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b93c8-c1f8-40ef-9753-05e63cdb3be5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:08.148852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1262/2516] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:08.150216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:08.161147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3d96c-988c-43b7-b84e-bc811b59f550 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:08.167124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:08.167837 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.168236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.168555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] PATH_INFO: `/v3/projects/b1855c89803e4c48b6d75a16ce901722/users/a1752877cb4d4198ac4ae6d62956a00b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.170568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1855c89803e4c48b6d75a16ce901722, user_id=a1752877cb4d4198ac4ae6d62956a00b, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:08.173265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:08.184496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1255/2517] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:08 2022] PUT /identity/v3/projects/b1855c89803e4c48b6d75a16ce901722/users/a1752877cb4d4198ac4ae6d62956a00b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:08.196071 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0f90-45db-4edb-8bbb-caac25438583 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.196509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0f90-45db-4edb-8bbb-caac25438583 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.196919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0f90-45db-4edb-8bbb-caac25438583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.226111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ffff0f90-45db-4edb-8bbb-caac25438583 None None] MFA Rules not processed for user `a1752877cb4d4198ac4ae6d62956a00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:08.227361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ffff0f90-45db-4edb-8bbb-caac25438583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:08.304753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1263/2518] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:08.315835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:08.317603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:08.374122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cd1ba-a914-41d3-bdda-fd2e4152b6b4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:08.377882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2142ebb-6524-4f17-99f0-d776d9b9e9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:08.378604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2142ebb-6524-4f17-99f0-d776d9b9e9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.378886 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2142ebb-6524-4f17-99f0-d776d9b9e9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.379325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2142ebb-6524-4f17-99f0-d776d9b9e9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.388842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2142ebb-6524-4f17-99f0-d776d9b9e9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:08.390294 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2142ebb-6524-4f17-99f0-d776d9b9e9d5 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:08.400347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1256/2519] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:08.465331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff0f90-45db-4edb-8bbb-caac25438583 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:08.526070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff0f90-45db-4edb-8bbb-caac25438583 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:08.530750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:08.531391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.531671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.531947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] PATH_INFO: `/v3/users/bc8f3032a6a24b87abaa406bd3df6a6d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.533201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8f3032a6a24b87abaa406bd3df6a6d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:08.535167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:08.581358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8f3032a6a24b87abaa406bd3df6a6d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:08.586977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:08.589074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8f3032a6a24b87abaa406bd3df6a6d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:08.596568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8f3032a6a24b87abaa406bd3df6a6d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:08.597205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8f3032a6a24b87abaa406bd3df6a6d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:08.609495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8f3032a6a24b87abaa406bd3df6a6d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:08.615118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1264/2520] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:08 2022] DELETE /identity/v3/users/bc8f3032a6a24b87abaa406bd3df6a6d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:08.619570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d355-c862-4655-84db-91a907e5279f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.619854 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d355-c862-4655-84db-91a907e5279f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.620134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d355-c862-4655-84db-91a907e5279f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.638796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-49b8d355-c862-4655-84db-91a907e5279f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:08.640091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-49b8d355-c862-4655-84db-91a907e5279f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:08.697726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1257/2521] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:08.707175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:08.708627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:08.765476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b2d16-3d7a-4b48-b5c3-b87f85a66412 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:08.769213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:08.769814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:08.770093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:08.770366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:08.779796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:08.781264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:08.790532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1265/2522] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:09.069780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8d355-c862-4655-84db-91a907e5279f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:09.071897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-49b8d355-c862-4655-84db-91a907e5279f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:09.085361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8d355-c862-4655-84db-91a907e5279f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:09.098099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:09.098099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:09.098099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:09.098099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] PATH_INFO: `/v3/projects/6cc8ee9f1d7842ebb755d2fd513fa859` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:09.098963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc8ee9f1d7842ebb755d2fd513fa859)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:09.098963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:09.143727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc8ee9f1d7842ebb755d2fd513fa859', 'request_id': 'req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.150979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc8ee9f1d7842ebb755d2fd513fa859', 'request_id': 'req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.156555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc8ee9f1d7842ebb755d2fd513fa859', 'request_id': 'req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.167370 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] The token cache is being invalidate because project 6cc8ee9f1d7842ebb755d2fd513fa859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:09.167659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.168354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.169978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1258/2523] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:09 2022] DELETE /identity/v3/projects/6cc8ee9f1d7842ebb755d2fd513fa859 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:09.173978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:09.184113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e57996-3cff-46ba-b8f2-c176d3cc18a7 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:09.237530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:09.238146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:09.238420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:09.238689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] PATH_INFO: `/v3/users/a1752877cb4d4198ac4ae6d62956a00b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:09.239838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1752877cb4d4198ac4ae6d62956a00b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:09.242284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:09.286714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1752877cb4d4198ac4ae6d62956a00b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.294128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1752877cb4d4198ac4ae6d62956a00b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.302201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1752877cb4d4198ac4ae6d62956a00b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.303388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1752877cb4d4198ac4ae6d62956a00b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.314024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:09.316602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f51e6f5e-3d34-4b79-aa04-739e5643ddde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1752877cb4d4198ac4ae6d62956a00b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.325129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1266/2524] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:09 2022] DELETE /identity/v3/users/a1752877cb4d4198ac4ae6d62956a00b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:09.595284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:09.596765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:09.614466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3eca1-121d-4e9b-95b7-c4c0b526b0d8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:09.619129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:09.674014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:09.674646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:09.674929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:09.675207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] PATH_INFO: `/v3/projects/b1855c89803e4c48b6d75a16ce901722` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:09.676347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1855c89803e4c48b6d75a16ce901722)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:09.678880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:09.722381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1855c89803e4c48b6d75a16ce901722', 'request_id': 'req-db00fcf5-fcc1-4342-903e-c2f460e21973'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.729969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1855c89803e4c48b6d75a16ce901722', 'request_id': 'req-db00fcf5-fcc1-4342-903e-c2f460e21973'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.735461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1855c89803e4c48b6d75a16ce901722', 'request_id': 'req-db00fcf5-fcc1-4342-903e-c2f460e21973'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.746658 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] The token cache is being invalidate because project b1855c89803e4c48b6d75a16ce901722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:09.746948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.747578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-db00fcf5-fcc1-4342-903e-c2f460e21973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:09.749070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1259/2525] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:09 2022] DELETE /identity/v3/projects/b1855c89803e4c48b6d75a16ce901722 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:09.753992 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fe5ca-070d-44e8-b42e-67dd8e7907b9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:09.754281 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fe5ca-070d-44e8-b42e-67dd8e7907b9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:09.754557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fe5ca-070d-44e8-b42e-67dd8e7907b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:09.774177 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a30fe5ca-070d-44e8-b42e-67dd8e7907b9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:09.775358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a30fe5ca-070d-44e8-b42e-67dd8e7907b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:09.832153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1267/2526] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:09.840438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af912a31-cf2e-4512-8874-178a4c907682 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:09.840677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af912a31-cf2e-4512-8874-178a4c907682 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:09.840939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af912a31-cf2e-4512-8874-178a4c907682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:09.860550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-af912a31-cf2e-4512-8874-178a4c907682 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:09.861731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-af912a31-cf2e-4512-8874-178a4c907682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:09.869440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1260/2527] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:09.873626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fe5ca-070d-44e8-b42e-67dd8e7907b9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:09.883163 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fe5ca-070d-44e8-b42e-67dd8e7907b9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:09.887902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:09.888534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:09.888815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:09.889087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:09.889729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:09.890029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:09.891617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:09.903702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1268/2528] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:08:09.907797 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af912a31-cf2e-4512-8874-178a4c907682 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:09.917522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af912a31-cf2e-4512-8874-178a4c907682 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:09.922856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:09.923446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:09.923718 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:09.923987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:09.924660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:09.926534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:09.953215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1261/2529] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:09 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:09.957597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:09.968622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484d173-4252-489e-b278-a1131691c20a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.023640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.024256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.024545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.024820 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.025441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.027344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.050284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1269/2530] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:09 2022] POST /identity/v3/users => generated 305 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:10.054377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.064512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd4dd4-515f-4063-bcc2-b87c60580c05 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.069228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.069822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.070100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.070377 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.071010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.072656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.082311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1262/2531] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:10.086547 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.096400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61fa4b-8b75-4f15-aeed-487857b797c9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.101605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51ace9-18f7-43ef-b19b-42362c4ec570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.102200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ace9-18f7-43ef-b19b-42362c4ec570 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.102477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ace9-18f7-43ef-b19b-42362c4ec570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.102748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ace9-18f7-43ef-b19b-42362c4ec570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.103389 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51ace9-18f7-43ef-b19b-42362c4ec570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.105016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51ace9-18f7-43ef-b19b-42362c4ec570 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.119428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1270/2532] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:10.120211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.130171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b588ec8-3ac1-431e-8149-d3719ddfbb92 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.135550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.136691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.136992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.137269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] PATH_INFO: `/v3/projects/62eb4cc3f841448da78a59934e904d09/users/8664c395d9a74c6ab97770bc6e385bc5/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.154088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62eb4cc3f841448da78a59934e904d09, user_id=8664c395d9a74c6ab97770bc6e385bc5, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.157047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.170246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1263/2533] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:10 2022] PUT /identity/v3/projects/62eb4cc3f841448da78a59934e904d09/users/8664c395d9a74c6ab97770bc6e385bc5/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:10.180926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f1eb-56a3-402a-847a-93163111c799 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.181237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f1eb-56a3-402a-847a-93163111c799 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.181543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f1eb-56a3-402a-847a-93163111c799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.210935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e404f1eb-56a3-402a-847a-93163111c799 None None] MFA Rules not processed for user `8664c395d9a74c6ab97770bc6e385bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:10.212169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e404f1eb-56a3-402a-847a-93163111c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:10.271890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:10.299942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1271/2534] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:10 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:10.305119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.340041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:10.372992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69810e9-6959-4e0a-ab48-45debb405944 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.377912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.378914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.378914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.379119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.379961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.381952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.425220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1264/2535] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:10 2022] POST /identity/v3/projects => generated 409 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:10.429881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404f1eb-56a3-402a-847a-93163111c799 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.441423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404f1eb-56a3-402a-847a-93163111c799 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.510461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.511338 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.511612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.511877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.512539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.514465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.540277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1272/2536] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:10 2022] POST /identity/v3/users => generated 311 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:10.542302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.554829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6dcdb-da54-453f-a25f-c0885e6449f5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.560573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.561190 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.561473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.561748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.562392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.564011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.575964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1265/2537] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:10.580191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.591032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40632022-258c-4e2f-b58f-f8e426a49a72 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.596087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.596782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.597061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.597328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] PATH_INFO: `/v3/projects/7dc80aff040b42b88e3a9486beef63ad/users/abd7193fd68f4efc986b6dc7043dd4a3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.612985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc80aff040b42b88e3a9486beef63ad, user_id=abd7193fd68f4efc986b6dc7043dd4a3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.615717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.628635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1273/2538] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:10 2022] PUT /identity/v3/projects/7dc80aff040b42b88e3a9486beef63ad/users/abd7193fd68f4efc986b6dc7043dd4a3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:10.633369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.643991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277df5e-ba91-465c-95c6-3d44b1105ee8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.700144 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd147f-c1fc-4f39-8cd9-175026116d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.700775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd147f-c1fc-4f39-8cd9-175026116d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.701059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd147f-c1fc-4f39-8cd9-175026116d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.701335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd147f-c1fc-4f39-8cd9-175026116d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.701970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd147f-c1fc-4f39-8cd9-175026116d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.703567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd147f-c1fc-4f39-8cd9-175026116d31 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.713531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1266/2539] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:10.717645 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.728333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa2968-63f7-45fc-adbb-9656dbbd046a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.732889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.733535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.733810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.734078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] PATH_INFO: `/v3/projects/7dc80aff040b42b88e3a9486beef63ad/users/abd7193fd68f4efc986b6dc7043dd4a3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.735967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc80aff040b42b88e3a9486beef63ad, user_id=abd7193fd68f4efc986b6dc7043dd4a3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.738638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.748919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1274/2540] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:10 2022] PUT /identity/v3/projects/7dc80aff040b42b88e3a9486beef63ad/users/abd7193fd68f4efc986b6dc7043dd4a3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:10.760110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f598c8-f545-402d-9d5b-b24d14404faa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.760445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f598c8-f545-402d-9d5b-b24d14404faa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.760724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f598c8-f545-402d-9d5b-b24d14404faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.789710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b1f598c8-f545-402d-9d5b-b24d14404faa None None] MFA Rules not processed for user `abd7193fd68f4efc986b6dc7043dd4a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:10.790925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b1f598c8-f545-402d-9d5b-b24d14404faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:10.855314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1267/2541] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:10 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:10.865886 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:10.867568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:10.924839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc37ea8-5557-4db1-891e-6b2b940e91be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:10.928532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:10.929134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:10.929409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:10.929680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:10.938714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:10.940141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:10.949456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1275/2542] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:10 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:11.102967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f598c8-f545-402d-9d5b-b24d14404faa None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:11.112783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f598c8-f545-402d-9d5b-b24d14404faa None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:11.116391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:11.116983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:11.117262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:11.117529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:11.144539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:11.177776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:11.179386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:11.189608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1268/2543] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:11 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:20.945276 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:21.184734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=15 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:21.322120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:21.323576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:21.377958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b178d3a-5270-4534-a42e-16bc1dce574a service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:21.381678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0329fc-1b19-41ac-9067-4aa6652b4bf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:21.382307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0329fc-1b19-41ac-9067-4aa6652b4bf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:21.382586 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0329fc-1b19-41ac-9067-4aa6652b4bf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:21.382858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0329fc-1b19-41ac-9067-4aa6652b4bf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:21.384585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dd0329fc-1b19-41ac-9067-4aa6652b4bf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:21.451627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0329fc-1b19-41ac-9067-4aa6652b4bf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:21.453101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0329fc-1b19-41ac-9067-4aa6652b4bf6 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:21.463456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1276/2544] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:21.615362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:21.684046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4483bd-bb31-4c68-a703-84424729783c service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:21.689154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:21.689785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:21.690060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:21.690332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] PATH_INFO: `/v3/users/8664c395d9a74c6ab97770bc6e385bc5` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:21.691446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8664c395d9a74c6ab97770bc6e385bc5)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:21.693903 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:21.739650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8664c395d9a74c6ab97770bc6e385bc5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:21.753060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8664c395d9a74c6ab97770bc6e385bc5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:21.760755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8664c395d9a74c6ab97770bc6e385bc5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:21.761727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8664c395d9a74c6ab97770bc6e385bc5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:21.762360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8664c395d9a74c6ab97770bc6e385bc5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:21.776594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1269/2545] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:21 2022] DELETE /identity/v3/users/8664c395d9a74c6ab97770bc6e385bc5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:21.781726 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:21.782020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:21.782322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:21.802429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:21.803725 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:21.869155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1277/2546] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:21.879462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:21.881562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:21.949773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492e352-72fd-420f-9136-7003d5d50447 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:21.954105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:21.954593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:21.954913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:21.955142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:21.966241 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:21.967852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:21.978357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1270/2547] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:22.713291 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:22.714797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:22.731333 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0021b-2a66-4e3c-a822-093d0d3ff004 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:22.736453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:22.737238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:22.737643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:22.738017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] PATH_INFO: `/v3/projects/62eb4cc3f841448da78a59934e904d09` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:22.739903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=62eb4cc3f841448da78a59934e904d09)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:22.742133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:22.804109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:22.808984 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62eb4cc3f841448da78a59934e904d09', 'request_id': 'req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:22.816792 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62eb4cc3f841448da78a59934e904d09', 'request_id': 'req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:22.823567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62eb4cc3f841448da78a59934e904d09', 'request_id': 'req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:22.839456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] The token cache is being invalidate because project 62eb4cc3f841448da78a59934e904d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:22.839879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:22.840618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:22.842411 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1278/2548] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:22 2022] DELETE /identity/v3/projects/62eb4cc3f841448da78a59934e904d09 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:22.846547 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:22.869546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3314cf7-bc6a-4603-b46b-ae40fbfff326 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:22.900090 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:22.951562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:22.952518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:22.952518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:22.952864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] PATH_INFO: `/v3/users/abd7193fd68f4efc986b6dc7043dd4a3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:22.955225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd7193fd68f4efc986b6dc7043dd4a3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:22.958630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:23.022185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd7193fd68f4efc986b6dc7043dd4a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.035612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd7193fd68f4efc986b6dc7043dd4a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.047612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd7193fd68f4efc986b6dc7043dd4a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.048381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd7193fd68f4efc986b6dc7043dd4a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.048990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd7193fd68f4efc986b6dc7043dd4a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.067602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1271/2549] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:22 2022] DELETE /identity/v3/users/abd7193fd68f4efc986b6dc7043dd4a3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:23.687641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:23.690420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:23.711598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0950888-c44b-4f29-bd3d-cb57ea0dab1b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:23.783614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:23.785841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:23.797351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:23.797988 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:23.798441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:23.799017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] PATH_INFO: `/v3/projects/7dc80aff040b42b88e3a9486beef63ad` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:23.800505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc80aff040b42b88e3a9486beef63ad)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:23.802835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:23.879250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc80aff040b42b88e3a9486beef63ad', 'request_id': 'req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.887437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc80aff040b42b88e3a9486beef63ad', 'request_id': 'req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.895234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc80aff040b42b88e3a9486beef63ad', 'request_id': 'req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.915663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818386f-8fe0-4f4c-8cb2-e44f7c7b02b3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:23.916158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] The token cache is being invalidate because project 7dc80aff040b42b88e3a9486beef63ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:23.916366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.917400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8b5fa02e-ad37-48cc-9cd2-0030b2597c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:23.922806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1279/2550] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:23 2022] DELETE /identity/v3/projects/7dc80aff040b42b88e3a9486beef63ad => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:23.924234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485f32-d08d-4727-9559-1f30d0449e2b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:23.924754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485f32-d08d-4727-9559-1f30d0449e2b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:23.925133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485f32-d08d-4727-9559-1f30d0449e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:23.956840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-48485f32-d08d-4727-9559-1f30d0449e2b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:23.958711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:23.962896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-48485f32-d08d-4727-9559-1f30d0449e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:23.992634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:23.993310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:23.993590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:23.993867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:23.995936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:24.026242 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.028472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.057534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1280/2551] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:24.066714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b984595-6285-4ce3-bb44-1fa6e0892f1e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.067227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b984595-6285-4ce3-bb44-1fa6e0892f1e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.067732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b984595-6285-4ce3-bb44-1fa6e0892f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.076622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1272/2552] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:24.079850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:24.104324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4b984595-6285-4ce3-bb44-1fa6e0892f1e None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:24.106653 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4b984595-6285-4ce3-bb44-1fa6e0892f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:24.122455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1281/2553] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:24.122838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.137548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa89705-880a-4ea6-8ad3-a1265aac226a service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.161960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.163557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.163557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.163557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.163882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:24.164101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.165716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.193036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b984595-6285-4ce3-bb44-1fa6e0892f1e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.198791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1273/2554] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:08:24.204708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b984595-6285-4ce3-bb44-1fa6e0892f1e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.209859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.210590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.210947 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.211408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.212468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.214611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.254545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.256386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1282/2555] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:24 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:24.269819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026db0-8210-4c84-bd11-c5469e5b3302 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.339908 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966960a-96e6-4930-882a-381272485448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.340839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966960a-96e6-4930-882a-381272485448 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.341119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966960a-96e6-4930-882a-381272485448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.341390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966960a-96e6-4930-882a-381272485448 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.342029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966960a-96e6-4930-882a-381272485448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.343955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966960a-96e6-4930-882a-381272485448 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.367556 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1274/2556] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:24 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:24.371675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.385910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90dd75-0929-42b8-b8f9-fdade6e14c02 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.391440 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.392129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.392534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.392876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.393609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.395323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.407918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1283/2557] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:24.412634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966960a-96e6-4930-882a-381272485448 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.422948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966960a-96e6-4930-882a-381272485448 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.427490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63b2c0-8843-4781-9d0f-9feb1ee9ee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.428082 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63b2c0-8843-4781-9d0f-9feb1ee9ee11 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.428409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63b2c0-8843-4781-9d0f-9feb1ee9ee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.428724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63b2c0-8843-4781-9d0f-9feb1ee9ee11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.429491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63b2c0-8843-4781-9d0f-9feb1ee9ee11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.431104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63b2c0-8843-4781-9d0f-9feb1ee9ee11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.442037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1275/2558] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:24.446560 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.457068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e93c-ec04-497f-833f-893346848c32 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.462613 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.463264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.463545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.463817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] PATH_INFO: `/v3/projects/cfa108c72def40bc945c90b6fd2f1068/users/f891cc877cb0434891da98cae0e1f140/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.480098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa108c72def40bc945c90b6fd2f1068, user_id=f891cc877cb0434891da98cae0e1f140, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.482817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.500364 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1284/2559] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:24 2022] PUT /identity/v3/projects/cfa108c72def40bc945c90b6fd2f1068/users/f891cc877cb0434891da98cae0e1f140/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:24.510058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc301d7d-4489-42a7-9801-baf299a9c2d9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.510345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc301d7d-4489-42a7-9801-baf299a9c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.510625 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc301d7d-4489-42a7-9801-baf299a9c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.544477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dc301d7d-4489-42a7-9801-baf299a9c2d9 None None] MFA Rules not processed for user `f891cc877cb0434891da98cae0e1f140`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:24.545915 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dc301d7d-4489-42a7-9801-baf299a9c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:24.636386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1276/2560] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:24.638084 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.741693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b103ba-146a-41fd-983b-d50d2ff82064 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.759253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.759899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.760179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.760492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.761394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.764620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.815573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1285/2561] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:24 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:24.820464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc301d7d-4489-42a7-9801-baf299a9c2d9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.834717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc301d7d-4489-42a7-9801-baf299a9c2d9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.863177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:24.914316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.914935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.915211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.915482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.916163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.918411 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:24.947035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1277/2562] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:24 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:24.952795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:24.967712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4faa83f-f5ba-4ce3-a0f5-9d494592cc1b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:24.973958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:24.974894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:24.975282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:24.975662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:24.976775 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:24.978800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:25.002805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1286/2563] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:25.004044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:25.020807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a167ac-b396-4520-b79c-b67828fd39a6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:25.038295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:25.038994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:25.039273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:25.039544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] PATH_INFO: `/v3/projects/ba0726a200bb42aea9891b6820e8e5e3/users/0e99444c0ae9405cb6a0cdc6703326ca/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:25.059822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0726a200bb42aea9891b6820e8e5e3, user_id=0e99444c0ae9405cb6a0cdc6703326ca, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:25.064000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:25.094847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1278/2564] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:25 2022] PUT /identity/v3/projects/ba0726a200bb42aea9891b6820e8e5e3/users/0e99444c0ae9405cb6a0cdc6703326ca/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:25.098829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:25.110512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ff834-5a29-4119-a3e2-db1e65839d7a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:25.191741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1b524-f964-46bb-ab63-3857665845ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:25.192528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b524-f964-46bb-ab63-3857665845ec admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:25.192871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b524-f964-46bb-ab63-3857665845ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:25.193756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b524-f964-46bb-ab63-3857665845ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:25.194500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1b524-f964-46bb-ab63-3857665845ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:25.196173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1b524-f964-46bb-ab63-3857665845ec admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:25.210858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1287/2565] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:25.211673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:25.222652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa333-9441-48b7-8a6e-08e8d495b230 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:25.230668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:25.231337 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:25.231616 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:25.231892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] PATH_INFO: `/v3/projects/ba0726a200bb42aea9891b6820e8e5e3/users/0e99444c0ae9405cb6a0cdc6703326ca/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:25.233869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0726a200bb42aea9891b6820e8e5e3, user_id=0e99444c0ae9405cb6a0cdc6703326ca, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:25.236623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:25.250896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1279/2566] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:25 2022] PUT /identity/v3/projects/ba0726a200bb42aea9891b6820e8e5e3/users/0e99444c0ae9405cb6a0cdc6703326ca/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:25.261472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1a7ba-b383-4dff-a6f3-8ad4e3862a22 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:25.261472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1a7ba-b383-4dff-a6f3-8ad4e3862a22 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:25.261786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1a7ba-b383-4dff-a6f3-8ad4e3862a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:25.289250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:25.306621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-14a1a7ba-b383-4dff-a6f3-8ad4e3862a22 None None] MFA Rules not processed for user `0e99444c0ae9405cb6a0cdc6703326ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:25.307953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14a1a7ba-b383-4dff-a6f3-8ad4e3862a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:25.388895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1288/2567] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:25.405156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:25.406958 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:25.468319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9323-9d20-433a-a766-a4a680e793f6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:25.472007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052e7aa-7a2b-4caa-9c11-6b155858c01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:25.472714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052e7aa-7a2b-4caa-9c11-6b155858c01d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:25.472925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052e7aa-7a2b-4caa-9c11-6b155858c01d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:25.473197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052e7aa-7a2b-4caa-9c11-6b155858c01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:25.484410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052e7aa-7a2b-4caa-9c11-6b155858c01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:25.485909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052e7aa-7a2b-4caa-9c11-6b155858c01d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:25.496153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1280/2568] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:26.074087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1a7ba-b383-4dff-a6f3-8ad4e3862a22 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:26.137120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1a7ba-b383-4dff-a6f3-8ad4e3862a22 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:26.141924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:26.142561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:26.142843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:26.143118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] PATH_INFO: `/v3/users/f891cc877cb0434891da98cae0e1f140` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:26.144329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f891cc877cb0434891da98cae0e1f140)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:26.146203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:26.203047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f891cc877cb0434891da98cae0e1f140'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.210181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f891cc877cb0434891da98cae0e1f140'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.218343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f891cc877cb0434891da98cae0e1f140'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.218999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f891cc877cb0434891da98cae0e1f140'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.231751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f891cc877cb0434891da98cae0e1f140'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.237853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1289/2569] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:26 2022] DELETE /identity/v3/users/f891cc877cb0434891da98cae0e1f140 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:26.243044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:26.243577 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:26.243703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:26.262539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:26.263654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:26.328774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1281/2570] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:26.339272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:26.340608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:26.358667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:26.399839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5f65-dd2a-48d4-9d19-0788e05fad2d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:26.403459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:26.404058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:26.404372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:26.404643 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:26.413446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:26.414877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:26.424081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1290/2571] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:26.684235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:26.685765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:26.698999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4ce69-5592-4e2d-99c0-b8fca0235265 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:26.703905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:26.704702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:26.705084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:26.705460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] PATH_INFO: `/v3/projects/cfa108c72def40bc945c90b6fd2f1068` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:26.706860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa108c72def40bc945c90b6fd2f1068)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:26.710720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:26.722781 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:26.784214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa108c72def40bc945c90b6fd2f1068', 'request_id': 'req-3d75f889-797c-4233-b9db-e64f7abf0233'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.792343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa108c72def40bc945c90b6fd2f1068', 'request_id': 'req-3d75f889-797c-4233-b9db-e64f7abf0233'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.798710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa108c72def40bc945c90b6fd2f1068', 'request_id': 'req-3d75f889-797c-4233-b9db-e64f7abf0233'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.815376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] The token cache is being invalidate because project cfa108c72def40bc945c90b6fd2f1068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:26.815604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.816401 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.818502 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1282/2572] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:26 2022] DELETE /identity/v3/projects/cfa108c72def40bc945c90b6fd2f1068 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:26.823540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:26.833115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5403d6-1156-41cf-aef7-e557267a6b21 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:26.890991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:26.891619 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:26.891899 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:26.892172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] PATH_INFO: `/v3/users/0e99444c0ae9405cb6a0cdc6703326ca` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:26.893353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e99444c0ae9405cb6a0cdc6703326ca)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:26.895522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:26.940770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e99444c0ae9405cb6a0cdc6703326ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.948149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e99444c0ae9405cb6a0cdc6703326ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.956476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e99444c0ae9405cb6a0cdc6703326ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.957253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e99444c0ae9405cb6a0cdc6703326ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.970870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-15fdd82d-3a4f-4742-87af-ea5d8bef66a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e99444c0ae9405cb6a0cdc6703326ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:26.976826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1291/2573] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:26 2022] DELETE /identity/v3/users/0e99444c0ae9405cb6a0cdc6703326ca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:27.255016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:27.257079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:27.274509 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75f889-797c-4233-b9db-e64f7abf0233 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:27.334818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:27.335479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.335763 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.336071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] PATH_INFO: `/v3/projects/ba0726a200bb42aea9891b6820e8e5e3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.337227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0726a200bb42aea9891b6820e8e5e3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:27.339167 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:27.389755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0726a200bb42aea9891b6820e8e5e3', 'request_id': 'req-40281f83-0532-4fc4-94ba-c6d754d98830'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:27.399285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0726a200bb42aea9891b6820e8e5e3', 'request_id': 'req-40281f83-0532-4fc4-94ba-c6d754d98830'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:27.406056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0726a200bb42aea9891b6820e8e5e3', 'request_id': 'req-40281f83-0532-4fc4-94ba-c6d754d98830'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:27.419982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] The token cache is being invalidate because project ba0726a200bb42aea9891b6820e8e5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:27.420446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:27.421306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40281f83-0532-4fc4-94ba-c6d754d98830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:27.422911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1283/2574] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:27 2022] DELETE /identity/v3/projects/ba0726a200bb42aea9891b6820e8e5e3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:27.428377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5d5d-57a3-4a86-b4d6-fc3d1ca5c784 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.428839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5d5d-57a3-4a86-b4d6-fc3d1ca5c784 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.428958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e5d5d-57a3-4a86-b4d6-fc3d1ca5c784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.450313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-601e5d5d-57a3-4a86-b4d6-fc3d1ca5c784 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:27.451535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-601e5d5d-57a3-4a86-b4d6-fc3d1ca5c784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:27.488123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:27.520152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1292/2575] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:27.527253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f348b-80d9-4777-a93e-e248893452af None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.527475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f348b-80d9-4777-a93e-e248893452af None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.527882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f348b-80d9-4777-a93e-e248893452af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.548705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-642f348b-80d9-4777-a93e-e248893452af None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:27.549889 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-642f348b-80d9-4777-a93e-e248893452af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:27.558303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1284/2576] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:27.562492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e5d5d-57a3-4a86-b4d6-fc3d1ca5c784 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:27.571957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e5d5d-57a3-4a86-b4d6-fc3d1ca5c784 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:27.576418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:27.577001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.577271 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.577536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.578171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:27.578465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:27.580032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:27.591175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1293/2577] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:08:27.595448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f348b-80d9-4777-a93e-e248893452af None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:27.604947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f348b-80d9-4777-a93e-e248893452af None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:27.609481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:27.610060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.610333 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.610602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.611223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:27.613133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:27.633706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:27.640912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1285/2578] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:27 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:27.644994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:27.654638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34dd1-e5a4-4be4-99e5-ba72e7546ec7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:27.709094 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:27.709723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.709996 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.710264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.710888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:27.712810 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:27.734017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1294/2579] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:27 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:27.738371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:27.749214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4c7ce-fc9f-497b-ab37-bc927dc375b8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:27.753832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:27.754418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.754696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.754970 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.755654 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:27.757289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:27.767617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1286/2580] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:27.771473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:27.781016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d9bef-1571-4755-beb8-09add8cb13da admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:27.785684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257cafd-7e49-41bb-91e0-badf3dea4f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:27.786272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257cafd-7e49-41bb-91e0-badf3dea4f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.786615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257cafd-7e49-41bb-91e0-badf3dea4f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.787462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257cafd-7e49-41bb-91e0-badf3dea4f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.788097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257cafd-7e49-41bb-91e0-badf3dea4f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:27.789772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257cafd-7e49-41bb-91e0-badf3dea4f5b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:27.800222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1295/2581] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:27.804722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:27.814434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a9856-ee84-4735-aafc-df85dd31b602 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:27.819287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:27.819963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.820267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.820552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] PATH_INFO: `/v3/projects/c7301d57fac84632aee4b8621ef101fe/users/bd4d991378c443e58cfa91395771a4d7/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.835152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7301d57fac84632aee4b8621ef101fe, user_id=bd4d991378c443e58cfa91395771a4d7, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:27.837888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:27.850838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1287/2582] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:27 2022] PUT /identity/v3/projects/c7301d57fac84632aee4b8621ef101fe/users/bd4d991378c443e58cfa91395771a4d7/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:27.915400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:27.915678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:27.915946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:27.944129 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] MFA Rules not processed for user `bd4d991378c443e58cfa91395771a4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:27.945346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:28.016925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1296/2583] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:28.022529 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.028169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.029554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:28.063778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:28.086989 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33989fd-a73e-4f73-8229-6f9bd131bd88 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.087246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a262c-4528-4f8f-ba9d-b497c58c1a31 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.092200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.092840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.093101 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.093341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.093542 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.094261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.094867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.095314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.095438 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:28.096278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.098681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.112510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.114385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.125689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1297/2584] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:28.131635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1288/2585] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:28 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:28.132812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.142753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e8e2b-520d-4a98-beb0-e65c80027072 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.200300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.201049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.201433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.201715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.202371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.204511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.227086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1298/2586] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:28 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:28.231158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.241745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02ae00-574c-4b25-9cf3-cd8d986ab326 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.246488 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.247083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.247358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.247626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.248276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.249856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.260848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1289/2587] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:28.266790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.277416 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea99cd-b7e8-4a29-8216-d7813347ee89 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.282124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.282847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.283200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.283480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] PATH_INFO: `/v3/projects/835079c03a5e481f95bc001b714c8d6f/users/31ce976a97f94f48878ae9338b3146c4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.299227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835079c03a5e481f95bc001b714c8d6f, user_id=31ce976a97f94f48878ae9338b3146c4, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.301952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.314944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1299/2588] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:28 2022] PUT /identity/v3/projects/835079c03a5e481f95bc001b714c8d6f/users/31ce976a97f94f48878ae9338b3146c4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:28.319580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.329771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddf463-5692-4a47-9738-89c94d9e347f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.386798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3493db-132b-44c9-a598-29347e3c3f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.387513 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3493db-132b-44c9-a598-29347e3c3f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.387798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3493db-132b-44c9-a598-29347e3c3f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.388065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3493db-132b-44c9-a598-29347e3c3f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.388735 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3493db-132b-44c9-a598-29347e3c3f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.390312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3493db-132b-44c9-a598-29347e3c3f9e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.400498 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1290/2589] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:28.405038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.415400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00f663-b8e3-4e70-a77c-b6932e5d7208 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.420112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.420793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.421072 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.421347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] PATH_INFO: `/v3/projects/835079c03a5e481f95bc001b714c8d6f/users/31ce976a97f94f48878ae9338b3146c4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.423258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] RBAC: Authorizing `identity:create_grant(project_id=835079c03a5e481f95bc001b714c8d6f, user_id=31ce976a97f94f48878ae9338b3146c4, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.425915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.436579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1300/2590] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:28 2022] PUT /identity/v3/projects/835079c03a5e481f95bc001b714c8d6f/users/31ce976a97f94f48878ae9338b3146c4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:28.447284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d27a-d95e-4ecd-9354-ae9233886011 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.447602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d27a-d95e-4ecd-9354-ae9233886011 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.447880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d27a-d95e-4ecd-9354-ae9233886011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.478682 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0542d27a-d95e-4ecd-9354-ae9233886011 None None] MFA Rules not processed for user `31ce976a97f94f48878ae9338b3146c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:28.479914 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0542d27a-d95e-4ecd-9354-ae9233886011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:28.507237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:28.568879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1291/2591] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:28.581212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.584335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:28.655936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08807df7-1d65-4fb7-81c9-6f8b4b03355c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.660548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.661161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.661445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.662188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.672761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.674322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.685336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1301/2592] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:28.772619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d27a-d95e-4ecd-9354-ae9233886011 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:28.783435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d27a-d95e-4ecd-9354-ae9233886011 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:28.789367 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da2a24-378c-4450-ac29-415d487fc849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:28.790441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da2a24-378c-4450-ac29-415d487fc849 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:28.790593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da2a24-378c-4450-ac29-415d487fc849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:28.791022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da2a24-378c-4450-ac29-415d487fc849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:28.860037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da2a24-378c-4450-ac29-415d487fc849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:28.861660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da2a24-378c-4450-ac29-415d487fc849 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:28.872041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1292/2593] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:29.179066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:29.246368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:29.291457 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedee64-e44c-48c5-b6c7-1afd54ccea7a service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:29.300081 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:29.300985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:29.301439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:29.302034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] PATH_INFO: `/v3/users/bd4d991378c443e58cfa91395771a4d7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:29.303959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4d991378c443e58cfa91395771a4d7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:29.306110 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:29.387328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4d991378c443e58cfa91395771a4d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:29.395418 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4d991378c443e58cfa91395771a4d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:29.407589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4d991378c443e58cfa91395771a4d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:29.408565 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4d991378c443e58cfa91395771a4d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:29.444322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4d991378c443e58cfa91395771a4d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:29.456859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1302/2594] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:29 2022] DELETE /identity/v3/users/bd4d991378c443e58cfa91395771a4d7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:29.469913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:29.470394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:29.470724 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:29.528062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:29.541006 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:29.679495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1293/2595] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:29.691909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:29.693953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:29.764353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9b052-9661-4cf3-9323-24bf82a4ba3c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:29.769844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:29.770733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:29.772712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:29.773264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:29.788050 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:29.789831 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:29.801044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1303/2596] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:30.215646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:30.217893 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:30.244373 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8053f-49d3-430b-8d22-9ac7804797c7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:30.259821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:30.260823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:30.261455 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:30.261825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] PATH_INFO: `/v3/projects/c7301d57fac84632aee4b8621ef101fe` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:30.264443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7301d57fac84632aee4b8621ef101fe)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:30.273311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:30.373916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7301d57fac84632aee4b8621ef101fe', 'request_id': 'req-ea1c05a3-0d48-433b-8036-6d612edb1ca4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.378153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:30.395475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7301d57fac84632aee4b8621ef101fe', 'request_id': 'req-ea1c05a3-0d48-433b-8036-6d612edb1ca4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.408489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7301d57fac84632aee4b8621ef101fe', 'request_id': 'req-ea1c05a3-0d48-433b-8036-6d612edb1ca4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.422514 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] The token cache is being invalidate because project c7301d57fac84632aee4b8621ef101fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:30.422900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.423643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.428467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1294/2597] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:30 2022] DELETE /identity/v3/projects/c7301d57fac84632aee4b8621ef101fe => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:30.429488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:30.441431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea367-5f3e-4fd2-b8ed-f0ebed4e20ba service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:30.516627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:30.531887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:30.533206 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:30.534435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:30.534782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] PATH_INFO: `/v3/users/31ce976a97f94f48878ae9338b3146c4` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:30.537046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ce976a97f94f48878ae9338b3146c4)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:30.539669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:30.603873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ce976a97f94f48878ae9338b3146c4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.615732 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ce976a97f94f48878ae9338b3146c4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.626378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ce976a97f94f48878ae9338b3146c4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.627159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ce976a97f94f48878ae9338b3146c4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.644615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f141ff8c-58e2-4fd9-82a8-c92b7d0d5d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ce976a97f94f48878ae9338b3146c4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:30.652286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1304/2598] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:30 2022] DELETE /identity/v3/users/31ce976a97f94f48878ae9338b3146c4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:31.188311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:31.190032 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:31.209422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c05a3-0d48-433b-8036-6d612edb1ca4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:31.303446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:31.304174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:31.304624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:31.304932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] PATH_INFO: `/v3/projects/835079c03a5e481f95bc001b714c8d6f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:31.306123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] RBAC: Authorizing `identity:delete_project(project_id=835079c03a5e481f95bc001b714c8d6f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:31.308068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:31.399855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835079c03a5e481f95bc001b714c8d6f', 'request_id': 'req-24da4210-1269-47ac-b9d7-2430e58f625f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:31.433636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835079c03a5e481f95bc001b714c8d6f', 'request_id': 'req-24da4210-1269-47ac-b9d7-2430e58f625f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:31.450585 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835079c03a5e481f95bc001b714c8d6f', 'request_id': 'req-24da4210-1269-47ac-b9d7-2430e58f625f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:31.471308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] The token cache is being invalidate because project 835079c03a5e481f95bc001b714c8d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:31.471465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:31.472507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-24da4210-1269-47ac-b9d7-2430e58f625f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:31.474158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1295/2599] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:31 2022] DELETE /identity/v3/projects/835079c03a5e481f95bc001b714c8d6f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:31.479182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da15a30-2d43-42f8-ab22-c43a676d185a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:31.479763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da15a30-2d43-42f8-ab22-c43a676d185a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:31.480371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da15a30-2d43-42f8-ab22-c43a676d185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:31.506052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8da15a30-2d43-42f8-ab22-c43a676d185a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:31.508758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8da15a30-2d43-42f8-ab22-c43a676d185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:31.633722 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1305/2600] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:31.644376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d0b8-e61f-42ba-a793-2455af048025 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:31.645087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d0b8-e61f-42ba-a793-2455af048025 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:31.645687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d0b8-e61f-42ba-a793-2455af048025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:31.683773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3a73d0b8-e61f-42ba-a793-2455af048025 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:31.684980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3a73d0b8-e61f-42ba-a793-2455af048025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:31.707796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1296/2601] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:31.713056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da15a30-2d43-42f8-ab22-c43a676d185a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:31.734674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da15a30-2d43-42f8-ab22-c43a676d185a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:31.745152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:31.745817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:31.746277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:31.746674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:31.752066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:31.759263 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:31.761716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:31.778277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1306/2602] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:08:31.782733 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d0b8-e61f-42ba-a793-2455af048025 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:31.793055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d0b8-e61f-42ba-a793-2455af048025 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:31.807442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:31.808792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:31.811901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:31.812203 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:31.813036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:31.815201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:31.854260 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1297/2603] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:31 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:31.858681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:31.870176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d9328-45e0-4bd8-9b9a-0e0090406ccf admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:31.944216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:31.944922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:31.945279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:31.945629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:31.946447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:31.948719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:31.960569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:31.989955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1307/2604] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:31 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:31.993966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.010290 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92003d1-0543-4363-ba3c-19837f3b0850 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.017515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.018422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.018937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.019920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.020922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.024523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.052140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1298/2605] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:32.057142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.073836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab591e4-1d59-49a2-b224-12a4843307e7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.081509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0526f11-d66c-40cf-b15f-8676525e1234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.082414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0526f11-d66c-40cf-b15f-8676525e1234 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.085115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0526f11-d66c-40cf-b15f-8676525e1234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.085757 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0526f11-d66c-40cf-b15f-8676525e1234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.087365 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0526f11-d66c-40cf-b15f-8676525e1234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.105031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0526f11-d66c-40cf-b15f-8676525e1234 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.122307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1308/2606] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:32.125046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.147061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b0a83-8a6f-4b39-bd22-b9010011dded admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.153369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.153953 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.154366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.154754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] PATH_INFO: `/v3/projects/cf132def3a7540da8dbb5c4dd81a826d/users/ed72f983fb3144c7a6ebe1eb16aebc7d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.180868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf132def3a7540da8dbb5c4dd81a826d, user_id=ed72f983fb3144c7a6ebe1eb16aebc7d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.184145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.208361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1299/2607] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:32 2022] PUT /identity/v3/projects/cf132def3a7540da8dbb5c4dd81a826d/users/ed72f983fb3144c7a6ebe1eb16aebc7d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:32.227324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6153c78-53e7-41cc-a744-f12c12557ffd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.227324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6153c78-53e7-41cc-a744-f12c12557ffd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.227324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6153c78-53e7-41cc-a744-f12c12557ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.268626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f6153c78-53e7-41cc-a744-f12c12557ffd None None] MFA Rules not processed for user `ed72f983fb3144c7a6ebe1eb16aebc7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:32.274228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f6153c78-53e7-41cc-a744-f12c12557ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.274703 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.276641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.294534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:32.371680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdef31-a90a-42a3-a95e-06f57949e444 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.372487 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1309/2608] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:32.376774 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.376927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6153c78-53e7-41cc-a744-f12c12557ffd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.377402 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.377678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.377971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] PATH_INFO: `/v3/users/85d2f3d2a8cb40d38d0ca2dc16609ef5` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.379062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d2f3d2a8cb40d38d0ca2dc16609ef5)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.381596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.388091 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6153c78-53e7-41cc-a744-f12c12557ffd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.395597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.396290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.396573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.396838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.397464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.399379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.426733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1310/2609] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:32.430017 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d2f3d2a8cb40d38d0ca2dc16609ef5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.431300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.441872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d2f3d2a8cb40d38d0ca2dc16609ef5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.449182 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d2f3d2a8cb40d38d0ca2dc16609ef5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.449820 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d2f3d2a8cb40d38d0ca2dc16609ef5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.450437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d2f3d2a8cb40d38d0ca2dc16609ef5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.463576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1300/2610] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:32 2022] DELETE /identity/v3/users/85d2f3d2a8cb40d38d0ca2dc16609ef5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:32.494858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbd418-d83b-4fa4-b49c-90f43d070021 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.543392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:32.553647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.554256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.554564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.554832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.555463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.557408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.578447 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1311/2611] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:32.582520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.584046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.598297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fa630-2ebc-4710-8647-d560dc674f97 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.603058 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.603656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.603937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.604211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.604948 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.606524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.617732 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1301/2612] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:32.621704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.633942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1792fc-5900-4258-b4d7-f56375a8096f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.640457 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.640971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.641805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.642105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.642273 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.642650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.643954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.647871 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.657002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583fa85-8031-485b-a6a9-e338cfc06d30 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.662532 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1312/2613] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:32.667178 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.667865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.668199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.668640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.668929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] PATH_INFO: `/v3/projects/4b0db8e5fc6c48cc98136ef2a85b05f4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.670710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0db8e5fc6c48cc98136ef2a85b05f4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.673124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.679102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad004-9e70-44a7-a45d-394ea22134ec admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.683813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27a1d-66c3-4c67-8580-758111d73649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.684484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27a1d-66c3-4c67-8580-758111d73649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.684760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27a1d-66c3-4c67-8580-758111d73649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.685654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27a1d-66c3-4c67-8580-758111d73649 admin admin] PATH_INFO: `/v3/projects/af9aee2b857b4248b5b6f10d5951b43f/users/231715bf59474c2bba3e4086f32dc6e4/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.701765 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27a1d-66c3-4c67-8580-758111d73649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9aee2b857b4248b5b6f10d5951b43f, user_id=231715bf59474c2bba3e4086f32dc6e4, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.704525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27a1d-66c3-4c67-8580-758111d73649 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.719040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1313/2614] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:32 2022] PUT /identity/v3/projects/af9aee2b857b4248b5b6f10d5951b43f/users/231715bf59474c2bba3e4086f32dc6e4/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:32.724064 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0db8e5fc6c48cc98136ef2a85b05f4', 'request_id': 'req-b5f310df-d27d-46e8-89fc-54e8c3cf2728'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.730698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.731061 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.731372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.734042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0db8e5fc6c48cc98136ef2a85b05f4', 'request_id': 'req-b5f310df-d27d-46e8-89fc-54e8c3cf2728'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.739928 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0db8e5fc6c48cc98136ef2a85b05f4', 'request_id': 'req-b5f310df-d27d-46e8-89fc-54e8c3cf2728'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.748216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:32.753783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] The token cache is being invalidate because project 4b0db8e5fc6c48cc98136ef2a85b05f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:32.753783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.759434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] MFA Rules not processed for user `231715bf59474c2bba3e4086f32dc6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:32.760454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5f310df-d27d-46e8-89fc-54e8c3cf2728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:32.760692 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.762310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1302/2615] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:32 2022] DELETE /identity/v3/projects/4b0db8e5fc6c48cc98136ef2a85b05f4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:32.767002 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c42a70-9250-41cd-b24b-5fbdfbe08d95 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.767287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c42a70-9250-41cd-b24b-5fbdfbe08d95 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.767571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c42a70-9250-41cd-b24b-5fbdfbe08d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.788879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b6c42a70-9250-41cd-b24b-5fbdfbe08d95 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:32.790074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b6c42a70-9250-41cd-b24b-5fbdfbe08d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.827612 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1314/2616] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:32.831999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.833576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.849657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1303/2617] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:32.857548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ec77a-2eba-497e-bca2-674ddf2d5b3f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.857843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ec77a-2eba-497e-bca2-674ddf2d5b3f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.858124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ec77a-2eba-497e-bca2-674ddf2d5b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.878734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f70ec77a-2eba-497e-bca2-674ddf2d5b3f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:32.879942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f70ec77a-2eba-497e-bca2-674ddf2d5b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:32.888125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1304/2618] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:32.892116 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ec77a-2eba-497e-bca2-674ddf2d5b3f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.900605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d888d-8122-4519-93a1-a35a91584fa7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.900880 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ec77a-2eba-497e-bca2-674ddf2d5b3f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.905120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.905544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.905759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.905983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.906109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.906272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.906380 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.906643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:32.906865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.907276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:32.907578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:32.908767 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.909179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:32.922962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1305/2619] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:08:32.928035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.936007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1315/2620] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:32.940008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:32.940805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ee7bd-87ec-4645-8af3-157aef1173c0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:32.965848 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:32.998246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:32.998854 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:32.999135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:32.999409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.000035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.002000 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.004502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ea09-15fb-43ad-8c34-982b7f8ebbb8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.009101 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.009693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.009967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.010237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.010981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.013179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.030171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1306/2621] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:33.034392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.034524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1316/2622] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:32 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:33.038739 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.048743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265a35e-f168-446e-a091-de27fd7f6df5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.100057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401eaa1c-2da7-404e-84ad-030dc94bf17c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.104753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.105145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.105282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.105434 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.105541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.105684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.105816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.105917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.106346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.106524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.108092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.108272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.117887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1317/2623] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:33.121253 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:33.121924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.131257 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1307/2624] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:33 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:33.132405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977691f-82a6-4795-a1c6-1ae5c789da1b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.135243 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.137468 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.138117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.138398 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.138674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] PATH_INFO: `/v3/projects/d3d5c72e73cf4d908f5ae5345e205859/users/1f624296378e458fa32d36fdcf8853b3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.145539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9026f-7644-4f57-a353-3d4a07b9f0fd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.150983 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.151942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.152525 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.152809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.153441 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.154002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d5c72e73cf4d908f5ae5345e205859, user_id=1f624296378e458fa32d36fdcf8853b3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.155008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.156664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.166214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1308/2625] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:33.170382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.170897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1318/2626] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:33 2022] PUT /identity/v3/projects/d3d5c72e73cf4d908f5ae5345e205859/users/1f624296378e458fa32d36fdcf8853b3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:33.174505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.182544 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125424a-395f-461f-aa21-4b98fd48436a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.238786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58247-9d70-40b7-83d6-267f75e4bfd4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.240501 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.241102 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.241376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.241642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.242270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.243839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.244144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.244860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.245217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.245682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.246396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.248046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.253982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1309/2627] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:33.257830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.260350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1319/2628] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:33.262186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.268980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb79e6-9ff9-4057-a72f-ce8e8da9e362 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.272422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e7ef-bbf3-423b-ba74-8bb1149f017b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.274398 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78459df5-6697-4d53-9545-5e846ec0cdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.275041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459df5-6697-4d53-9545-5e846ec0cdd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.275325 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459df5-6697-4d53-9545-5e846ec0cdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.275605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78459df5-6697-4d53-9545-5e846ec0cdd0 admin admin] PATH_INFO: `/v3/projects/ba46e33140ea42d9bda6144dbda37f07/users/86aa47eb9a3a42908173e0031d46d695/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.277190 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00894c45-a755-4063-bda6-63af03220a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.277828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894c45-a755-4063-bda6-63af03220a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.278107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894c45-a755-4063-bda6-63af03220a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.278380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894c45-a755-4063-bda6-63af03220a82 admin admin] PATH_INFO: `/v3/projects/d3d5c72e73cf4d908f5ae5345e205859/users/1f624296378e458fa32d36fdcf8853b3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.280371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00894c45-a755-4063-bda6-63af03220a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d5c72e73cf4d908f5ae5345e205859, user_id=1f624296378e458fa32d36fdcf8853b3, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.283045 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00894c45-a755-4063-bda6-63af03220a82 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.291152 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78459df5-6697-4d53-9545-5e846ec0cdd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba46e33140ea42d9bda6144dbda37f07, user_id=86aa47eb9a3a42908173e0031d46d695, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.294157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78459df5-6697-4d53-9545-5e846ec0cdd0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.301055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1320/2629] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:33 2022] PUT /identity/v3/projects/d3d5c72e73cf4d908f5ae5345e205859/users/1f624296378e458fa32d36fdcf8853b3/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:33.307743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1310/2630] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:33 2022] PUT /identity/v3/projects/ba46e33140ea42d9bda6144dbda37f07/users/86aa47eb9a3a42908173e0031d46d695/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:33.312182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.312507 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.312785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.317977 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.318329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.318671 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.326921 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:33.343144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] MFA Rules not processed for user `1f624296378e458fa32d36fdcf8853b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:33.344388 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:33.348001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] MFA Rules not processed for user `86aa47eb9a3a42908173e0031d46d695`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:33.349363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:33.420572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1311/2631] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:33.426749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1321/2632] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:33.432832 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.434491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:33.440282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.441800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:33.502848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:33.513077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102b86-aabd-42ad-99fb-6ae776a3dc17 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.517087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5884940-9bc7-45ef-885f-a251b69f08bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.518169 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5884940-9bc7-45ef-885f-a251b69f08bb service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.518745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5884940-9bc7-45ef-885f-a251b69f08bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.519026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5884940-9bc7-45ef-885f-a251b69f08bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.520898 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d5884940-9bc7-45ef-885f-a251b69f08bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:33.540827 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115530a6-5f7a-4b44-b31a-2f5068f3ab80 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.548394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.549158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.549534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.549898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.560134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.561891 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.573271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1312/2633] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:33.614948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5884940-9bc7-45ef-885f-a251b69f08bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.617610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5884940-9bc7-45ef-885f-a251b69f08bb service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.632034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1322/2634] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:33.808397 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:33.879010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5fecc-09c6-43e2-9c33-18f0553af0e2 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:33.892994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:33.893685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:33.893980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:33.894258 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] PATH_INFO: `/v3/users/86aa47eb9a3a42908173e0031d46d695` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:33.895955 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] RBAC: Authorizing `identity:delete_user(user_id=86aa47eb9a3a42908173e0031d46d695)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:33.897709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:33.962755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86aa47eb9a3a42908173e0031d46d695'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:33.976986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86aa47eb9a3a42908173e0031d46d695'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:33.984073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86aa47eb9a3a42908173e0031d46d695'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:33.984902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86aa47eb9a3a42908173e0031d46d695'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:33.985504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86aa47eb9a3a42908173e0031d46d695'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:33.999683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1313/2635] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:33 2022] DELETE /identity/v3/users/86aa47eb9a3a42908173e0031d46d695 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:34.003972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.004471 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.004574 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.024212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:34.025424 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:34.064930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:34.082436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1323/2636] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:34.092268 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.093672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:34.150473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938a90d-0423-44cd-a62b-fb5dae29a31a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.154140 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5ad24-65cd-423c-a19e-d304368f5373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.154737 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ad24-65cd-423c-a19e-d304368f5373 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.155013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ad24-65cd-423c-a19e-d304368f5373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.155283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ad24-65cd-423c-a19e-d304368f5373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.161060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:34.166098 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5ad24-65cd-423c-a19e-d304368f5373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.167748 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5ad24-65cd-423c-a19e-d304368f5373 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.178985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1314/2637] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:34.452838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.454251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:34.466833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b5b55-a526-4d7e-919b-dddde7381e2f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.472120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.472958 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.473318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.474874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] PATH_INFO: `/v3/projects/ba46e33140ea42d9bda6144dbda37f07` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.476317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba46e33140ea42d9bda6144dbda37f07)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.478355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.525210 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba46e33140ea42d9bda6144dbda37f07', 'request_id': 'req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:34.531397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba46e33140ea42d9bda6144dbda37f07', 'request_id': 'req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:34.537633 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba46e33140ea42d9bda6144dbda37f07', 'request_id': 'req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:34.549528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] The token cache is being invalidate because project ba46e33140ea42d9bda6144dbda37f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:34.549811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:34.550412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-fe7fc6d8-35bb-468f-b8a8-be1b720736c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:34.551892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1324/2638] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:34 2022] DELETE /identity/v3/projects/ba46e33140ea42d9bda6144dbda37f07 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:34.558308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9337-cc89-425f-8e6a-558b01c8df7a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.558776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9337-cc89-425f-8e6a-558b01c8df7a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.558900 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9337-cc89-425f-8e6a-558b01c8df7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.579461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b4ee9337-cc89-425f-8e6a-558b01c8df7a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:34.580734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b4ee9337-cc89-425f-8e6a-558b01c8df7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:34.639081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1315/2639] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:34.648480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922695-223a-4405-91b3-0d1dc900c2dd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.648860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922695-223a-4405-91b3-0d1dc900c2dd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.649141 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922695-223a-4405-91b3-0d1dc900c2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.669408 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5e922695-223a-4405-91b3-0d1dc900c2dd None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:34.670595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e922695-223a-4405-91b3-0d1dc900c2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:34.678837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1325/2640] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:34.682753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9337-cc89-425f-8e6a-558b01c8df7a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.692469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9337-cc89-425f-8e6a-558b01c8df7a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.697232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.697825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.698105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.698374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.699007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:34.699302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.700952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.717059 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1316/2641] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:08:34.718090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e922695-223a-4405-91b3-0d1dc900c2dd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.727901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e922695-223a-4405-91b3-0d1dc900c2dd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.733082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.733697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.733977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.734250 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.735259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.737847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.766526 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1326/2642] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:34 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:34.770535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.781269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1f431-ec12-4cae-9bf3-0744a9797c18 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.845307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.845923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.846204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.846482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.847122 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.849836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.871407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1317/2643] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:34 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:34.876184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.886905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba7a5-8bd1-41dc-98bd-bd0cc63669b2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.891495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.892086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.892394 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.892663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.893298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.894874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.906208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1327/2644] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:34.910446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.921438 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184fbf6-1c88-4511-b16e-60ae35b9ee1c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.926133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6115d48-be03-49d2-b388-bf8055b7f17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.926818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6115d48-be03-49d2-b388-bf8055b7f17b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.927239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6115d48-be03-49d2-b388-bf8055b7f17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.927582 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6115d48-be03-49d2-b388-bf8055b7f17b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.928223 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6115d48-be03-49d2-b388-bf8055b7f17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.929878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6115d48-be03-49d2-b388-bf8055b7f17b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.940271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1318/2645] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:34.944493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:34.954230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddddf86-150b-44bb-9af1-b262ad458281 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:34.960528 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:34.961218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:34.961499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:34.961782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] PATH_INFO: `/v3/projects/609d02459706428c9eb319e255b2a96d/users/206105969a364f96b7091602611cda1b/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:34.978420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609d02459706428c9eb319e255b2a96d, user_id=206105969a364f96b7091602611cda1b, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:34.981159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:34.994979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1328/2646] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:34 2022] PUT /identity/v3/projects/609d02459706428c9eb319e255b2a96d/users/206105969a364f96b7091602611cda1b/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:35.006645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.006930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.007202 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.036305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] MFA Rules not processed for user `206105969a364f96b7091602611cda1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:35.037534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:35.085353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:35.103619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1319/2647] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:35.108201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.175225 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.175683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc9c8-567b-42c5-bbd7-b93e51901a92 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.176640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:35.180555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.181160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.181441 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.181713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.182351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.184370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.196346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:35.214358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1329/2648] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:35 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:35.218288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.236603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbe4f1-cfb2-42c8-ae1a-3269b0b53c19 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.283858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d600da4-cd1e-42a6-ad1b-e893f9bd4bbf admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.288771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.289568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.289855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.290149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.290787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.292845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.292991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.293555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.293828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.294100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.295926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:35.315222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1330/2649] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:35 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:35.319977 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.331484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1babaf-3211-4184-b983-5684d3914266 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.336844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.337463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.337740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.338014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.338652 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.340235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.350295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1331/2650] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:35.354380 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.363484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.363790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b23f-9204-42fb-b3e3-b239afc5ab8a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.365470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.370269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.370930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.371200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.371477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] PATH_INFO: `/v3/projects/b789766f759742eabdce62132317d793/users/e9970b4a1f9340ea9bc325ec2067bcef/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.375267 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1320/2651] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:35.389426 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b789766f759742eabdce62132317d793, user_id=e9970b4a1f9340ea9bc325ec2067bcef, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.392152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.407402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1332/2652] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:35 2022] PUT /identity/v3/projects/b789766f759742eabdce62132317d793/users/e9970b4a1f9340ea9bc325ec2067bcef/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:35.411601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.422153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29fd03-5091-423f-84e7-9b3b199e5a6d service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.478043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9d6a3-ddb4-436f-87f1-18f978699ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.478641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9d6a3-ddb4-436f-87f1-18f978699ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.478906 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9d6a3-ddb4-436f-87f1-18f978699ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.479211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9d6a3-ddb4-436f-87f1-18f978699ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.479807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9d6a3-ddb4-436f-87f1-18f978699ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.481433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9d6a3-ddb4-436f-87f1-18f978699ce5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.482690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:35.494137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1321/2653] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:35.498312 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.509919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f10d4b-44b1-4f5b-9ae8-6992f69ee215 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.514909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.515585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.515865 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.516143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] PATH_INFO: `/v3/projects/b789766f759742eabdce62132317d793/users/e9970b4a1f9340ea9bc325ec2067bcef/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.518098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b789766f759742eabdce62132317d793, user_id=e9970b4a1f9340ea9bc325ec2067bcef, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.520783 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.532080 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1333/2654] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:35 2022] PUT /identity/v3/projects/b789766f759742eabdce62132317d793/users/e9970b4a1f9340ea9bc325ec2067bcef/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:35.543542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6226e4c-8813-461f-bf78-7a14a0f32600 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.543786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6226e4c-8813-461f-bf78-7a14a0f32600 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.544067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6226e4c-8813-461f-bf78-7a14a0f32600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.575617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e6226e4c-8813-461f-bf78-7a14a0f32600 None None] MFA Rules not processed for user `e9970b4a1f9340ea9bc325ec2067bcef`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:35.577019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6226e4c-8813-461f-bf78-7a14a0f32600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:35.652105 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1322/2655] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:35.662960 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.664693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:35.728222 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6b9d1-ddd4-42c9-bd00-80762508b7ed admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.731980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.732658 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.733001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.733252 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.743405 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:35.744938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:35.755891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1334/2656] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:35.954301 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6226e4c-8813-461f-bf78-7a14a0f32600 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:35.966054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6226e4c-8813-461f-bf78-7a14a0f32600 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:35.971448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:35.972038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:35.972374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:35.972738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:35.975951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:36.047689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:36.049909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:36.067092 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:36.068510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:36.079963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1323/2657] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:36.101168 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:36.125476 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62754-52e3-4868-b294-d54d9464e873 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:36.129454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320426a-eaea-4e0d-ae40-9325e0983f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:36.130134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320426a-eaea-4e0d-ae40-9325e0983f35 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:36.130409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320426a-eaea-4e0d-ae40-9325e0983f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:36.130681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320426a-eaea-4e0d-ae40-9325e0983f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:36.132628 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f320426a-eaea-4e0d-ae40-9325e0983f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:36.210968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320426a-eaea-4e0d-ae40-9325e0983f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:36.212454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320426a-eaea-4e0d-ae40-9325e0983f35 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:36.223136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1335/2658] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:37.213126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:37.234515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48c5a-4d66-48f1-b025-f06459f9ac55 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:37.240231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:37.240767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:37.241049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:37.241327 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] PATH_INFO: `/v3/users/206105969a364f96b7091602611cda1b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:37.242487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206105969a364f96b7091602611cda1b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:37.244689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:37.313021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206105969a364f96b7091602611cda1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:37.334061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206105969a364f96b7091602611cda1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:37.339901 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:37.345119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206105969a364f96b7091602611cda1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:37.347689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206105969a364f96b7091602611cda1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:37.348373 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206105969a364f96b7091602611cda1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:37.363248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1324/2659] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:37 2022] DELETE /identity/v3/users/206105969a364f96b7091602611cda1b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:37.368905 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:37.369600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:37.369887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:37.395386 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:37.396609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:37.492769 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1336/2660] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:37.504909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:37.507248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:37.589783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadfd59-6bcd-4ef0-bc48-26a5d62c3e95 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:37.593736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:37.594611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:37.594894 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:37.595168 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:37.606711 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:37.608146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:37.618596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1325/2661] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:37.912913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:37.926616 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:37.945772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79a6cf-75c0-4fd6-93fb-ae45aa6d6278 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:37.952985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:37.953809 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:37.954112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:37.954393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] PATH_INFO: `/v3/projects/609d02459706428c9eb319e255b2a96d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:37.955608 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609d02459706428c9eb319e255b2a96d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:37.960634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:38.027794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609d02459706428c9eb319e255b2a96d', 'request_id': 'req-a1b63209-846b-4580-aaed-491948e6ee87'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.035320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609d02459706428c9eb319e255b2a96d', 'request_id': 'req-a1b63209-846b-4580-aaed-491948e6ee87'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.042203 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609d02459706428c9eb319e255b2a96d', 'request_id': 'req-a1b63209-846b-4580-aaed-491948e6ee87'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.057506 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] The token cache is being invalidate because project 609d02459706428c9eb319e255b2a96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:38.057959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.058657 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.060362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1337/2662] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:37 2022] DELETE /identity/v3/projects/609d02459706428c9eb319e255b2a96d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:38.064518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:38.075357 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6bee7-8228-4962-bb90-a1351386cc69 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:38.139684 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:38.140329 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:38.140630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:38.141104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] PATH_INFO: `/v3/users/e9970b4a1f9340ea9bc325ec2067bcef` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:38.142899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9970b4a1f9340ea9bc325ec2067bcef)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:38.145414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:38.198176 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9970b4a1f9340ea9bc325ec2067bcef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.213794 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9970b4a1f9340ea9bc325ec2067bcef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.221935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9970b4a1f9340ea9bc325ec2067bcef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.222773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9970b4a1f9340ea9bc325ec2067bcef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.223443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6ff68bc4-9684-4ed0-9693-b8588f6be4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9970b4a1f9340ea9bc325ec2067bcef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.242163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1326/2663] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:38 2022] DELETE /identity/v3/users/e9970b4a1f9340ea9bc325ec2067bcef => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:38.566537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:38.568086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:38.589566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b63209-846b-4580-aaed-491948e6ee87 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:38.595037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:38.664530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:38.665175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:38.665449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:38.665811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] PATH_INFO: `/v3/projects/b789766f759742eabdce62132317d793` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:38.666954 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b789766f759742eabdce62132317d793)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:38.668982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:38.723818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b789766f759742eabdce62132317d793', 'request_id': 'req-6814642a-c09f-47ee-b382-5aa57706b53d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.732236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b789766f759742eabdce62132317d793', 'request_id': 'req-6814642a-c09f-47ee-b382-5aa57706b53d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.742655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b789766f759742eabdce62132317d793', 'request_id': 'req-6814642a-c09f-47ee-b382-5aa57706b53d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.759262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] The token cache is being invalidate because project b789766f759742eabdce62132317d793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:38.759642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.760342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6814642a-c09f-47ee-b382-5aa57706b53d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:38.762103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1338/2664] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:38 2022] DELETE /identity/v3/projects/b789766f759742eabdce62132317d793 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:38.769635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec620b9-89aa-4c87-853b-6562d8b14eaf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:38.770145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec620b9-89aa-4c87-853b-6562d8b14eaf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:38.770618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec620b9-89aa-4c87-853b-6562d8b14eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:38.796807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3ec620b9-89aa-4c87-853b-6562d8b14eaf None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:38.796807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3ec620b9-89aa-4c87-853b-6562d8b14eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:38.804479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:38.896351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1327/2665] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:38.908479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682aaf-fdc7-4a63-8746-67585c41c163 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:38.908479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682aaf-fdc7-4a63-8746-67585c41c163 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:38.908479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682aaf-fdc7-4a63-8746-67585c41c163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:38.929859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f9682aaf-fdc7-4a63-8746-67585c41c163 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:38.931090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f9682aaf-fdc7-4a63-8746-67585c41c163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:38.949590 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1339/2666] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:38.950595 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec620b9-89aa-4c87-853b-6562d8b14eaf None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:38.963595 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec620b9-89aa-4c87-853b-6562d8b14eaf None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:38.986911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:38.987622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:38.988057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:38.988382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:38.990252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:38.990413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:38.992143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.012446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1328/2667] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:08:39.017892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682aaf-fdc7-4a63-8746-67585c41c163 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.030684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682aaf-fdc7-4a63-8746-67585c41c163 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.038344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.039323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.039698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.040057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.040843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.044948 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.079827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1340/2668] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:39 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:39.093635 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.105878 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0f183-ef73-4d0f-917d-009efefdf52b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.187489 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.188579 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.189067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.189771 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.190847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.195707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.245271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1329/2669] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:39 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:39.247194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.268181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb033d-a4b5-4bc7-8fdc-3d526d283d49 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.279335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.280056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.280674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.280964 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.281605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.291640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.298161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1341/2670] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:39.304640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.320785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c23af-ce82-4a8d-a73b-2b2e18221f4c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.327379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a593ad5-c5ef-4a94-9dcc-136fa1c34450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.328562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a593ad5-c5ef-4a94-9dcc-136fa1c34450 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.329214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a593ad5-c5ef-4a94-9dcc-136fa1c34450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.329881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a593ad5-c5ef-4a94-9dcc-136fa1c34450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.331148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a593ad5-c5ef-4a94-9dcc-136fa1c34450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.333913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a593ad5-c5ef-4a94-9dcc-136fa1c34450 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.359533 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1330/2671] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:39.364014 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.375795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd35d-4a63-459a-be35-d5f90e705578 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.381559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.382360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.382720 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.383125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] PATH_INFO: `/v3/projects/ac1a30c7e0e443f2a30c9fe660daac1c/users/d402cfe2503b4f1d8e0d2e0db7126fd3/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.399942 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac1a30c7e0e443f2a30c9fe660daac1c, user_id=d402cfe2503b4f1d8e0d2e0db7126fd3, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.402876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.423714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1342/2672] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:39 2022] PUT /identity/v3/projects/ac1a30c7e0e443f2a30c9fe660daac1c/users/d402cfe2503b4f1d8e0d2e0db7126fd3/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:39.435449 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4509ab-9f7c-4c3e-a697-de08bd7ff389 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.435752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4509ab-9f7c-4c3e-a697-de08bd7ff389 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.435984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4509ab-9f7c-4c3e-a697-de08bd7ff389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.489685 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ec4509ab-9f7c-4c3e-a697-de08bd7ff389 None None] MFA Rules not processed for user `d402cfe2503b4f1d8e0d2e0db7126fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:39.491407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ec4509ab-9f7c-4c3e-a697-de08bd7ff389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:39.572024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1331/2673] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:39.579617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.626056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:39.661143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7c417-9341-4571-8811-0373ec00eb2b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.666631 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.667286 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.668142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.668142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.668636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.670591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.700311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1343/2674] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:39 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:39.704822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4509ab-9f7c-4c3e-a697-de08bd7ff389 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.712127 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:39.718003 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4509ab-9f7c-4c3e-a697-de08bd7ff389 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.791670 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.792350 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.792663 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.793477 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.793656 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.795697 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.826054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1332/2675] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:39 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:39.827615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.837623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.840289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:39.842789 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167533e-f69f-4b6f-9d89-e110fe7e7ff3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.848098 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.848275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.848519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.849695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.850189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.852736 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.855483 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbaae3-c0d0-4ffe-a382-2c91f0a0b410 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.860288 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.860931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.861207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.861486 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] PATH_INFO: `/v3/users/a06de37941264b979a3bd5aaf84ec73c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.862621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06de37941264b979a3bd5aaf84ec73c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.864562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.864832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1344/2676] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:39.869036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.878763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c1d3d-a0b3-4596-9945-43845bcbbdb4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:39.883915 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:39.884581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:39.884860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:39.885130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] PATH_INFO: `/v3/projects/61002e91f2684232957b54deb6183255/users/add837c7d7c84ace8ce7b58b6dafacb9/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:39.905575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=61002e91f2684232957b54deb6183255, user_id=add837c7d7c84ace8ce7b58b6dafacb9, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:39.908596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:39.919756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06de37941264b979a3bd5aaf84ec73c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:39.922437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1345/2677] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:39 2022] PUT /identity/v3/projects/61002e91f2684232957b54deb6183255/users/add837c7d7c84ace8ce7b58b6dafacb9/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:39.926261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:39.932730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06de37941264b979a3bd5aaf84ec73c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:39.939994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06de37941264b979a3bd5aaf84ec73c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:39.940817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06de37941264b979a3bd5aaf84ec73c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:39.941496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06de37941264b979a3bd5aaf84ec73c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:39.954300 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1333/2678] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:39 2022] DELETE /identity/v3/users/a06de37941264b979a3bd5aaf84ec73c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:39.990115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179583-1744-4561-b72a-89f9a2f390aa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.047840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c949c1-e45f-4c72-88c2-aec33dad87f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.048404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c949c1-e45f-4c72-88c2-aec33dad87f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.048683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c949c1-e45f-4c72-88c2-aec33dad87f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.048950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c949c1-e45f-4c72-88c2-aec33dad87f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.049581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c949c1-e45f-4c72-88c2-aec33dad87f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.051194 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c949c1-e45f-4c72-88c2-aec33dad87f4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.060763 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1346/2679] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:40.064972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.066423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.079882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61577813-251d-4efc-a903-ea9851d9286f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.085237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.085924 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.086263 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.086662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] PATH_INFO: `/v3/projects/61002e91f2684232957b54deb6183255/users/add837c7d7c84ace8ce7b58b6dafacb9/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.088777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61002e91f2684232957b54deb6183255, user_id=add837c7d7c84ace8ce7b58b6dafacb9, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.091426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.105259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1334/2680] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:40 2022] PUT /identity/v3/projects/61002e91f2684232957b54deb6183255/users/add837c7d7c84ace8ce7b58b6dafacb9/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:40.118872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.118872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.118872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.147665 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.150076 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.166133 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] MFA Rules not processed for user `add837c7d7c84ace8ce7b58b6dafacb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:40.167279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.194896 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:40.221675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:40.236298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b94d-4bfd-4e08-902e-582666e1e70b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.241812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.242534 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.242888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.243235 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] PATH_INFO: `/v3/projects/b9d96d936ce8417197008a59da394df7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.244731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d96d936ce8417197008a59da394df7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.246667 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.256800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1347/2681] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:40.264439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.264439 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.273128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca26b7-1615-44e6-829c-390e7612a82d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.277968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.278587 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.278866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.279140 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] PATH_INFO: `/v3/users/ed72f983fb3144c7a6ebe1eb16aebc7d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.280275 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed72f983fb3144c7a6ebe1eb16aebc7d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.282175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.315621 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d96d936ce8417197008a59da394df7', 'request_id': 'req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.324602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d96d936ce8417197008a59da394df7', 'request_id': 'req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.330335 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d96d936ce8417197008a59da394df7', 'request_id': 'req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.337654 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed72f983fb3144c7a6ebe1eb16aebc7d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.342124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] The token cache is being invalidate because project b9d96d936ce8417197008a59da394df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:40.342413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.343071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.344627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1335/2682] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:40 2022] DELETE /identity/v3/projects/b9d96d936ce8417197008a59da394df7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:40.344811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed72f983fb3144c7a6ebe1eb16aebc7d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.345404 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.346681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.354576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed72f983fb3144c7a6ebe1eb16aebc7d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.355351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed72f983fb3144c7a6ebe1eb16aebc7d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.367838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-04767a1f-77ca-4a32-a15f-34084e787dc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed72f983fb3144c7a6ebe1eb16aebc7d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.374095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1348/2683] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:40 2022] DELETE /identity/v3/users/ed72f983fb3144c7a6ebe1eb16aebc7d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:40.375581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0514c8-377f-4997-b535-97809d866c67 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.375872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0514c8-377f-4997-b535-97809d866c67 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.376147 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0514c8-377f-4997-b535-97809d866c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.396500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bd0514c8-377f-4997-b535-97809d866c67 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:40.397737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd0514c8-377f-4997-b535-97809d866c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.410716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5ccad-6c49-45a7-bf81-e98b53bd4222 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.458324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1349/2684] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:40.460027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84028-2b45-4b40-9034-c5bdcef4f302 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.460355 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84028-2b45-4b40-9034-c5bdcef4f302 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.460695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84028-2b45-4b40-9034-c5bdcef4f302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.465872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.466496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.466779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.467053 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.468964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.480597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3cc84028-2b45-4b40-9034-c5bdcef4f302 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:40.481772 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3cc84028-2b45-4b40-9034-c5bdcef4f302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.489585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1350/2685] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:40.490941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.491516 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.491978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.515774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:40.516979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.520693 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:40.525873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1351/2686] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:40.530053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.531358 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.538362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.539799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.545577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:40.550108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1336/2687] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:40.550943 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.561819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64805978-eb61-4c67-8765-8c3f07ba0715 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.567027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.567626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.568422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.568694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.569332 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:40.569722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.575454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.593932 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1337/2688] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:08:40.595423 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.605782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6782d60-5370-4b1f-a6bd-8d966cc679e5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.609220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a5989-7113-419b-87bf-10374563ccaf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.611496 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.612121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.612611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.612978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.613134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.613591 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.613713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.613965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.614499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.616088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.626686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.628540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.641872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1352/2689] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:40.656852 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1338/2690] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:40.661009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.672811 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22afe-7a75-48f0-a481-5724ce323e62 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.700156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.702284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.748934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.749698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.750060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.750436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.751097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.753522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.772960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c1f31-e6fb-49d5-a3af-85c7667a78d3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.780379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1353/2691] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:40.781390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.781560 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.782280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.782412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.782639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] PATH_INFO: `/v3/users/d402cfe2503b4f1d8e0d2e0db7126fd3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.783800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d402cfe2503b4f1d8e0d2e0db7126fd3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.785764 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.793390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9c4b8-df5b-4eb2-9625-2e322bf50609 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.805020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.807893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.808040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.808413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.809135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.811125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.830796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1354/2692] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:40.834403 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.850715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5738503-99cf-44fd-b2b2-bf242e7cc889 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.862641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.863464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.863836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.864406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.865122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.865248 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd402cfe2503b4f1d8e0d2e0db7126fd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.866854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.879715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1355/2693] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:40.884115 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:40.884602 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd402cfe2503b4f1d8e0d2e0db7126fd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.894207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd402cfe2503b4f1d8e0d2e0db7126fd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.895070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd402cfe2503b4f1d8e0d2e0db7126fd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.895750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dfc51-8208-4b79-8402-88735a241d2b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:40.895929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8d7751f-9a40-45ed-a6ff-31911aed9724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd402cfe2503b4f1d8e0d2e0db7126fd3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:40.901419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfc25c-1e8a-4dbe-9590-fd7f1d1b2d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:40.902315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc25c-1e8a-4dbe-9590-fd7f1d1b2d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.902665 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc25c-1e8a-4dbe-9590-fd7f1d1b2d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.902978 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc25c-1e8a-4dbe-9590-fd7f1d1b2d22 admin admin] PATH_INFO: `/v3/projects/0ef67d6c6e08422893f0174953f24e0d/users/a70c32d0b5c847c4b34a1cbb494db37e/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.911572 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1339/2694] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:40 2022] DELETE /identity/v3/users/d402cfe2503b4f1d8e0d2e0db7126fd3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:40.916355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.916829 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.917204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.923533 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfc25c-1e8a-4dbe-9590-fd7f1d1b2d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef67d6c6e08422893f0174953f24e0d, user_id=a70c32d0b5c847c4b34a1cbb494db37e, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:40.926227 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfc25c-1e8a-4dbe-9590-fd7f1d1b2d22 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:40.938541 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:40.939913 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:40.942112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1356/2695] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:40 2022] PUT /identity/v3/projects/0ef67d6c6e08422893f0174953f24e0d/users/a70c32d0b5c847c4b34a1cbb494db37e/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:40.953821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:40.954248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:40.954598 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:40.985618 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:40.988118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] MFA Rules not processed for user `a70c32d0b5c847c4b34a1cbb494db37e`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:40.989446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.027831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1340/2696] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:41.038826 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.040356 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.043117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:41.065903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1357/2697] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:41.077228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.078659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.111313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356954b-cd92-4294-9d3d-3801907bf11b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:41.115911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:41.115911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:41.116201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:41.116355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:41.126727 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:41.128165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:41.139845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1341/2698] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:41.160912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb29263-7089-4294-a34e-25009c76f6c9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:41.168903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95b365b4794c4edebe306e09336c42f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:41.169641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:41.170361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:41.170361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:41.185877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:41.187943 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:41.200571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.201569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1358/2699] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:41.202693 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.218016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30828d-c563-40b3-ab59-25f80bacb91e service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:41.224014 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:41.224753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:41.225044 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:41.225298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] PATH_INFO: `/v3/projects/cf132def3a7540da8dbb5c4dd81a826d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:41.226470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf132def3a7540da8dbb5c4dd81a826d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:41.228388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:41.289617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf132def3a7540da8dbb5c4dd81a826d', 'request_id': 'req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.299551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf132def3a7540da8dbb5c4dd81a826d', 'request_id': 'req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.306554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf132def3a7540da8dbb5c4dd81a826d', 'request_id': 'req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.325754 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] The token cache is being invalidate because project cf132def3a7540da8dbb5c4dd81a826d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:41.326023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.327662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.333043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1342/2700] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:41 2022] DELETE /identity/v3/projects/cf132def3a7540da8dbb5c4dd81a826d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:41.337550 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.348064 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3ea11-b987-4575-8e17-d5797cd2cf76 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:41.419982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:41.420640 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:41.420914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:41.421193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] PATH_INFO: `/v3/users/231715bf59474c2bba3e4086f32dc6e4` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:41.422493 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] RBAC: Authorizing `identity:delete_user(user_id=231715bf59474c2bba3e4086f32dc6e4)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:41.424469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:41.469857 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.471742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.497445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8bfb-2ef0-44f9-9b29-8feba9d9fd01 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:41.498261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231715bf59474c2bba3e4086f32dc6e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.503071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:41.503822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:41.504183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:41.504559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] PATH_INFO: `/v3/projects/ac1a30c7e0e443f2a30c9fe660daac1c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:41.506125 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac1a30c7e0e443f2a30c9fe660daac1c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:41.508675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:41.508825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231715bf59474c2bba3e4086f32dc6e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.517017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231715bf59474c2bba3e4086f32dc6e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.517859 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231715bf59474c2bba3e4086f32dc6e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.530051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:41.533761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231715bf59474c2bba3e4086f32dc6e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.548419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1359/2701] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:41 2022] DELETE /identity/v3/users/231715bf59474c2bba3e4086f32dc6e4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:41.576369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac1a30c7e0e443f2a30c9fe660daac1c', 'request_id': 'req-0015a314-05a3-4002-b0c2-208ca2dabfec'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.582035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:41.589123 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac1a30c7e0e443f2a30c9fe660daac1c', 'request_id': 'req-0015a314-05a3-4002-b0c2-208ca2dabfec'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.599578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac1a30c7e0e443f2a30c9fe660daac1c', 'request_id': 'req-0015a314-05a3-4002-b0c2-208ca2dabfec'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.625721 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] The token cache is being invalidate because project ac1a30c7e0e443f2a30c9fe660daac1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:41.626018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.626882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.629413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1343/2702] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:41 2022] DELETE /identity/v3/projects/ac1a30c7e0e443f2a30c9fe660daac1c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:41.635930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.637667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.721817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3f897-aea1-46bc-bf35-334921e8492a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:41.781746 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:41.782371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:41.782644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:41.782913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] PATH_INFO: `/v3/users/add837c7d7c84ace8ce7b58b6dafacb9` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:41.784492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add837c7d7c84ace8ce7b58b6dafacb9)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:41.786024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:41.833537 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add837c7d7c84ace8ce7b58b6dafacb9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.841855 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add837c7d7c84ace8ce7b58b6dafacb9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.850264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.852403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.854524 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add837c7d7c84ace8ce7b58b6dafacb9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.855274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add837c7d7c84ace8ce7b58b6dafacb9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.867722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015a314-05a3-4002-b0c2-208ca2dabfec admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:41.871224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add837c7d7c84ace8ce7b58b6dafacb9'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.872743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:41.873789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:41.873789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:41.874269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] PATH_INFO: `/v3/projects/af9aee2b857b4248b5b6f10d5951b43f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:41.875485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9aee2b857b4248b5b6f10d5951b43f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:41.877497 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:41.877697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1360/2703] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:41 2022] DELETE /identity/v3/users/add837c7d7c84ace8ce7b58b6dafacb9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:41.928765 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9aee2b857b4248b5b6f10d5951b43f', 'request_id': 'req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.937659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9aee2b857b4248b5b6f10d5951b43f', 'request_id': 'req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.945749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9aee2b857b4248b5b6f10d5951b43f', 'request_id': 'req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.959299 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] The token cache is being invalidate because project af9aee2b857b4248b5b6f10d5951b43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:41.959650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.960413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:41.962623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1344/2704] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:41 2022] DELETE /identity/v3/projects/af9aee2b857b4248b5b6f10d5951b43f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:41.966890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:41.968361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:41.993572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:42.042985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e5491-4e9e-4fe5-8c05-14d7a45eba17 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.102059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.102695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.102973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.103247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] PATH_INFO: `/v3/users/1f624296378e458fa32d36fdcf8853b3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.104427 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f624296378e458fa32d36fdcf8853b3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.106362 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.154119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f624296378e458fa32d36fdcf8853b3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.161112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f624296378e458fa32d36fdcf8853b3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.168225 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f624296378e458fa32d36fdcf8853b3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.168917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f624296378e458fa32d36fdcf8853b3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.179518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.181128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:42.188867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f334c2f3-687b-4e24-a8b7-09ee25382d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f624296378e458fa32d36fdcf8853b3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.198719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1361/2705] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:41 2022] DELETE /identity/v3/users/1f624296378e458fa32d36fdcf8853b3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:42.200162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13eea6-4ed1-4b2e-b3be-0bec2c0fc9b8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.266109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.266783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.267063 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.267343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] PATH_INFO: `/v3/projects/61002e91f2684232957b54deb6183255` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.268677 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61002e91f2684232957b54deb6183255)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.271093 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.324527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61002e91f2684232957b54deb6183255', 'request_id': 'req-0d761b4c-1089-46ea-a517-1d4307b9981a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.332545 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61002e91f2684232957b54deb6183255', 'request_id': 'req-0d761b4c-1089-46ea-a517-1d4307b9981a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.338417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61002e91f2684232957b54deb6183255', 'request_id': 'req-0d761b4c-1089-46ea-a517-1d4307b9981a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.340407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:42.365054 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] The token cache is being invalidate because project 61002e91f2684232957b54deb6183255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:42.365358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.366074 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d761b4c-1089-46ea-a517-1d4307b9981a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.367592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1345/2706] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:42 2022] DELETE /identity/v3/projects/61002e91f2684232957b54deb6183255 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:42.372462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.372753 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.373030 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.394825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:42.396267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:42.440093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:42.458917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1362/2707] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:42.466644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca1fcc-6f81-4710-99ae-e395d92048cc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.466935 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca1fcc-6f81-4710-99ae-e395d92048cc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.467210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca1fcc-6f81-4710-99ae-e395d92048cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.487255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1aca1fcc-6f81-4710-99ae-e395d92048cc None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:42.488630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1aca1fcc-6f81-4710-99ae-e395d92048cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:42.491983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.493481 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:42.497555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1346/2708] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:42.502024 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca1fcc-6f81-4710-99ae-e395d92048cc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.507894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac8acf-6940-4614-bf5a-947cea363642 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.513008 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca1fcc-6f81-4710-99ae-e395d92048cc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.514236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.515461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.515829 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.516184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] PATH_INFO: `/v3/projects/d3d5c72e73cf4d908f5ae5345e205859` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.517929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d5c72e73cf4d908f5ae5345e205859)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.518049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.518974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.519374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.519707 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.519839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.520636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:42.521087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.522919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.535083 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1347/2709] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:08:42.540204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.550487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aaa92d-f1f1-44d2-98ea-c37d207b014e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.555865 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.556999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.557470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.557920 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.559035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.562605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.579392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d5c72e73cf4d908f5ae5345e205859', 'request_id': 'req-1190c2f2-9b0a-4448-a1a2-29864279a1f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.587392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d5c72e73cf4d908f5ae5345e205859', 'request_id': 'req-1190c2f2-9b0a-4448-a1a2-29864279a1f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.592470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1348/2710] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:42 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:42.593259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d5c72e73cf4d908f5ae5345e205859', 'request_id': 'req-1190c2f2-9b0a-4448-a1a2-29864279a1f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.597162 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.606601 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] The token cache is being invalidate because project d3d5c72e73cf4d908f5ae5345e205859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:42.607003 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.607662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:42.609681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1363/2711] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:42 2022] DELETE /identity/v3/projects/d3d5c72e73cf4d908f5ae5345e205859 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:42.667259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea759694-106f-42c7-902e-99c6d3e32672 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.672454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.673088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.673371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.673649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.674302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.676345 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.699887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1349/2712] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:42 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:42.704154 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.705819 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:42.791321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c2f2-9b0a-4448-a1a2-29864279a1f5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.796162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.796825 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.797104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.797377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.798065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.799682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.813849 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1364/2713] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:42.818049 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.829104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1672eb-2ebd-4170-b004-28e3cad9a755 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.834647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45700bb2-2c45-4d8b-9c5a-8ea3a2b97324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.835348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45700bb2-2c45-4d8b-9c5a-8ea3a2b97324 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.835753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45700bb2-2c45-4d8b-9c5a-8ea3a2b97324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.836034 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45700bb2-2c45-4d8b-9c5a-8ea3a2b97324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.836795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45700bb2-2c45-4d8b-9c5a-8ea3a2b97324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.838484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45700bb2-2c45-4d8b-9c5a-8ea3a2b97324 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.850107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1350/2714] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:42.861814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:42.873557 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4011-6247-49ae-9282-f8be4a5822fd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:42.878842 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:42.879728 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.879965 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.880282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] PATH_INFO: `/v3/projects/e6ba55ea2518413e81eebb7df7477dca/users/12413144eaf5425cb29367dbec1ee189/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.897221 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ba55ea2518413e81eebb7df7477dca, user_id=12413144eaf5425cb29367dbec1ee189, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:42.899868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:42.923095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1365/2715] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:42 2022] PUT /identity/v3/projects/e6ba55ea2518413e81eebb7df7477dca/users/12413144eaf5425cb29367dbec1ee189/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:42.925056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b4758-e964-43d5-bcf1-063bf99bd72a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:42.925346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b4758-e964-43d5-bcf1-063bf99bd72a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:42.925624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b4758-e964-43d5-bcf1-063bf99bd72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:42.954938 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-568b4758-e964-43d5-bcf1-063bf99bd72a None None] MFA Rules not processed for user `12413144eaf5425cb29367dbec1ee189`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:42.955564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-568b4758-e964-43d5-bcf1-063bf99bd72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:43.024767 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1351/2716] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:43.029315 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.073505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:43.091636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9550533-1563-4079-a064-02416e8391e7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.096230 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.096845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.097119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.097390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.098012 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.100491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.127280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1366/2717] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:43 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:43.131432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b4758-e964-43d5-bcf1-063bf99bd72a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.143433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b4758-e964-43d5-bcf1-063bf99bd72a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.162520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:43.203623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.204306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.204655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.204990 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.205554 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.207725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.228607 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1352/2718] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:43 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:43.232648 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.243200 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8c608-d37b-4943-91b1-cbd8aa0705ae admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.247822 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.248448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.248724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.248995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.249682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.251477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.262951 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1367/2719] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:43.266862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.276648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a772-c2aa-44cb-ad79-001969af95e6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.281170 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6ada9-e5d8-45f5-9be4-0c0af33c79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.281762 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ada9-e5d8-45f5-9be4-0c0af33c79bd admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.282039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ada9-e5d8-45f5-9be4-0c0af33c79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.282309 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6ada9-e5d8-45f5-9be4-0c0af33c79bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.282933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6ada9-e5d8-45f5-9be4-0c0af33c79bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.284538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6ada9-e5d8-45f5-9be4-0c0af33c79bd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.294976 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1353/2720] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:43.299056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.309891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cab02-e94d-4c6c-ab99-3e60355b03ea admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.315617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.316290 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.316575 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.316853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] PATH_INFO: `/v3/projects/0a049f222f0740a79e367a1ecf2b9a93/users/c8e37ad0de7448ad8d93e9433fc5699d/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.331231 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a049f222f0740a79e367a1ecf2b9a93, user_id=c8e37ad0de7448ad8d93e9433fc5699d, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.333870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.346082 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1368/2721] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:43 2022] PUT /identity/v3/projects/0a049f222f0740a79e367a1ecf2b9a93/users/c8e37ad0de7448ad8d93e9433fc5699d/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:43.357535 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808d7a-b420-4ef3-a042-717d5fede8cc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.358035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808d7a-b420-4ef3-a042-717d5fede8cc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.358156 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808d7a-b420-4ef3-a042-717d5fede8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.385646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0f808d7a-b420-4ef3-a042-717d5fede8cc None None] MFA Rules not processed for user `c8e37ad0de7448ad8d93e9433fc5699d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:43.387119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0f808d7a-b420-4ef3-a042-717d5fede8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:43.449087 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1354/2722] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:43.453627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.514035 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107f4a-b756-4293-95b8-6bb0b5580a3f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.518629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.519220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.519499 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.519771 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.520425 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.522443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.548962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1369/2723] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:43 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:43.553619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f808d7a-b420-4ef3-a042-717d5fede8cc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.564095 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f808d7a-b420-4ef3-a042-717d5fede8cc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.619199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.619858 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.620147 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.620460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.621106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.623651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.651802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1355/2724] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:43 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:43.652985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.663827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791767a-fbf3-4d07-8660-39c0befe1af4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.668448 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.669043 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.669324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.669599 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.670234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.671824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.682062 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1370/2725] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:43.686066 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.695749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298660d-70a8-4286-9e19-6963199ea86c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.700361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.700999 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.701275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.701550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] PATH_INFO: `/v3/projects/236b140acc384a46b7c23782981ba5db/users/744df4dd94b945b1b3333e919f5a785d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.716086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=236b140acc384a46b7c23782981ba5db, user_id=744df4dd94b945b1b3333e919f5a785d, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.718747 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.721722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:43.733297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1356/2726] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:43 2022] PUT /identity/v3/projects/236b140acc384a46b7c23782981ba5db/users/744df4dd94b945b1b3333e919f5a785d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:43.737153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.747102 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d0d0e-3bbe-4c81-bbca-c319808bb9b9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.802544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbaba0b-2cad-4921-8a55-2756c7b2c2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.803135 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbaba0b-2cad-4921-8a55-2756c7b2c2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.803412 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbaba0b-2cad-4921-8a55-2756c7b2c2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.803683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbaba0b-2cad-4921-8a55-2756c7b2c2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.804349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbaba0b-2cad-4921-8a55-2756c7b2c2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.805934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbaba0b-2cad-4921-8a55-2756c7b2c2d7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.815370 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1371/2727] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:08:43.819597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.829699 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30448764-fcea-47ea-b6d0-3070c2e61682 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:43.834834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:43.835473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.835753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.836028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] PATH_INFO: `/v3/projects/236b140acc384a46b7c23782981ba5db/users/744df4dd94b945b1b3333e919f5a785d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.837973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=236b140acc384a46b7c23782981ba5db, user_id=744df4dd94b945b1b3333e919f5a785d, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:43.840612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:43.851183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1357/2728] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:43 2022] PUT /identity/v3/projects/236b140acc384a46b7c23782981ba5db/users/744df4dd94b945b1b3333e919f5a785d/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:43.861253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:43.861535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:43.861807 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:43.868919 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:43.895006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] MFA Rules not processed for user `744df4dd94b945b1b3333e919f5a785d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:43.896718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:43.970477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1372/2729] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:43.980773 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:43.982447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.041806 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd327-2f1d-4c94-87f5-9cf9d57b1320 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.047526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.047828 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.048103 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.048403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.048788 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:44.050156 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.057879 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.059286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.069790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1358/2730] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:44.111110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:44.112587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.119256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dcdee-503a-4578-b3c5-8fa74508286b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.123652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab507-6652-4455-a795-db0a52592b13 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.123911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.124569 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.124846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.125152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] PATH_INFO: `/v3/users/a70c32d0b5c847c4b34a1cbb494db37e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.126261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70c32d0b5c847c4b34a1cbb494db37e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.127495 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe309fe-b005-48b8-a30e-f373d69fb883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.128077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe309fe-b005-48b8-a30e-f373d69fb883 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.128189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.128382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe309fe-b005-48b8-a30e-f373d69fb883 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.128650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe309fe-b005-48b8-a30e-f373d69fb883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.176353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70c32d0b5c847c4b34a1cbb494db37e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.183850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70c32d0b5c847c4b34a1cbb494db37e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.187760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe309fe-b005-48b8-a30e-f373d69fb883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.189304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe309fe-b005-48b8-a30e-f373d69fb883 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.191321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70c32d0b5c847c4b34a1cbb494db37e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.192059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70c32d0b5c847c4b34a1cbb494db37e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.198620 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1359/2731] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:44.204876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70c32d0b5c847c4b34a1cbb494db37e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.211318 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1373/2732] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:44 2022] DELETE /identity/v3/users/a70c32d0b5c847c4b34a1cbb494db37e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:44.215892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.216183 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.216504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.236029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:44.237277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.299013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1360/2733] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:44.309564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:44.311044 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.321429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:44.373335 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11224a-1d62-4479-86c4-f42be16f5092 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.377334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e6478-15b9-4c46-8ff4-99939c302e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.378021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6478-15b9-4c46-8ff4-99939c302e34 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.378308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6478-15b9-4c46-8ff4-99939c302e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.378577 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6478-15b9-4c46-8ff4-99939c302e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.388818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e6478-15b9-4c46-8ff4-99939c302e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.390245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e6478-15b9-4c46-8ff4-99939c302e34 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.401454 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1374/2734] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:44.683219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:44.685342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.698239 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b03cfe-65be-4ba2-b494-808e39659e71 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.702963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.703690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.703870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.704150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] PATH_INFO: `/v3/projects/0ef67d6c6e08422893f0174953f24e0d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.705324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef67d6c6e08422893f0174953f24e0d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.707334 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.730238 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:44.756331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef67d6c6e08422893f0174953f24e0d', 'request_id': 'req-a58de4e6-70ba-4a78-b8c7-5d813a086083'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.764207 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef67d6c6e08422893f0174953f24e0d', 'request_id': 'req-a58de4e6-70ba-4a78-b8c7-5d813a086083'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.770381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef67d6c6e08422893f0174953f24e0d', 'request_id': 'req-a58de4e6-70ba-4a78-b8c7-5d813a086083'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.782650 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] The token cache is being invalidate because project 0ef67d6c6e08422893f0174953f24e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:44.782939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.783591 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:44.785126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1361/2735] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:44 2022] DELETE /identity/v3/projects/0ef67d6c6e08422893f0174953f24e0d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:44.790410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d3a5f-fbee-4519-8b47-728b7350d3f4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.790815 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d3a5f-fbee-4519-8b47-728b7350d3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.790970 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d3a5f-fbee-4519-8b47-728b7350d3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.794481 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:44.795849 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.810525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2a8d3a5f-fbee-4519-8b47-728b7350d3f4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:44.812371 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2a8d3a5f-fbee-4519-8b47-728b7350d3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.866442 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58de4e6-70ba-4a78-b8c7-5d813a086083 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.871611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.872234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.872530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.872667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1375/2736] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:44.876487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.878302 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.880935 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8a82-a4d1-44f6-995e-e58050d219cd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.881220 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8a82-a4d1-44f6-995e-e58050d219cd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.881497 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8a82-a4d1-44f6-995e-e58050d219cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.891316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.892949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.901363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-39ce8a82-a4d1-44f6-995e-e58050d219cd None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:44.902551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-39ce8a82-a4d1-44f6-995e-e58050d219cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:44.903283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1362/2737] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:08:44.911738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1376/2738] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:08:44.916135 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:44.926161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed8b71-122b-4bf0-8bd6-21e15904ad0e service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.930659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.931237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.931512 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.931783 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.932437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:44.932742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.934305 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.946007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1363/2739] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:08:44.950132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8a82-a4d1-44f6-995e-e58050d219cd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:44.960549 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8a82-a4d1-44f6-995e-e58050d219cd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:44.965800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:44.966417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:44.966696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:44.966969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:44.967609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:44.969553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:44.997422 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1377/2740] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:44 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:45.001609 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.012905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac7c78-388b-4f4a-8600-44895f6ee274 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.071939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.072573 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.072848 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.073119 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.073746 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.075633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.096919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1364/2741] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:45 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:45.101162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.111719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b994-3fcf-49b5-95ab-308860af0458 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.117383 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.117980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.118261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.118534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.119172 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.120901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.131240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1378/2742] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:45.135106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.145385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa16926-78be-4b2f-a6f6-ef700a2bc979 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.150094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694df06-657d-41d8-b7b0-6a647139f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.150678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694df06-657d-41d8-b7b0-6a647139f5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.151026 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694df06-657d-41d8-b7b0-6a647139f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.151303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694df06-657d-41d8-b7b0-6a647139f5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.151963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694df06-657d-41d8-b7b0-6a647139f5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.153567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694df06-657d-41d8-b7b0-6a647139f5e7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.163815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1365/2743] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:45.167913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.178296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92add41e-6589-415c-a238-e243a7dd28c3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.182892 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.183551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.183850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.184148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] PATH_INFO: `/v3/projects/f9b6aff01dcb4dcfb97bbc4a8b7950c0/users/48dae07e9c5e4e73827a49fd47553955/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.199078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b6aff01dcb4dcfb97bbc4a8b7950c0, user_id=48dae07e9c5e4e73827a49fd47553955, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.201707 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.214717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1379/2744] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:45 2022] PUT /identity/v3/projects/f9b6aff01dcb4dcfb97bbc4a8b7950c0/users/48dae07e9c5e4e73827a49fd47553955/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:45.226284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0228c71-b590-44a0-90a9-32af0a348e8b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.226570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0228c71-b590-44a0-90a9-32af0a348e8b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.226839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0228c71-b590-44a0-90a9-32af0a348e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.233988 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:45.256725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c0228c71-b590-44a0-90a9-32af0a348e8b None None] MFA Rules not processed for user `48dae07e9c5e4e73827a49fd47553955`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:45.257933 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c0228c71-b590-44a0-90a9-32af0a348e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:45.323287 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1366/2745] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:45.327883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.374568 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:45.390580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d13bb-bf21-4a31-ba87-361e80b32b68 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.395212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.395834 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.396117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.396419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.397049 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.399005 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.426635 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1380/2746] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:45 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:45.431246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0228c71-b590-44a0-90a9-32af0a348e8b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.442046 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0228c71-b590-44a0-90a9-32af0a348e8b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.499841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.500472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.500753 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.501022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.501668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.503571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.525972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1367/2747] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:45 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:08:45.530090 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.540773 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027292ea-6ab3-4ed6-a521-8fdd50ba3934 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.545357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.545944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.546217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.546484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.547113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.548719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.558914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1381/2748] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:45.563115 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.574435 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75104125-b540-4f25-861a-39b8bbe4aa90 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.579594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.580601 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.580962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.581228 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] PATH_INFO: `/v3/projects/0c82ee17361740c9b919501bce700183/users/2dd31acd837b42708ab2bd050d5b550c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.598472 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c82ee17361740c9b919501bce700183, user_id=2dd31acd837b42708ab2bd050d5b550c, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.601681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.624415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1368/2749] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:45 2022] PUT /identity/v3/projects/0c82ee17361740c9b919501bce700183/users/2dd31acd837b42708ab2bd050d5b550c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:45.624535 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.631843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ae0a-b778-44e3-a708-c33282eed070 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.840182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bcd4e-b013-4547-b9a3-5e5e0cd78038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.840880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bcd4e-b013-4547-b9a3-5e5e0cd78038 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.841676 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bcd4e-b013-4547-b9a3-5e5e0cd78038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.842095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bcd4e-b013-4547-b9a3-5e5e0cd78038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.842832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bcd4e-b013-4547-b9a3-5e5e0cd78038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.844742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bcd4e-b013-4547-b9a3-5e5e0cd78038 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.858694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1382/2750] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:45.862751 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:45.875789 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b535d1-2fb9-4284-a2d1-e21c1603b7d7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:45.884419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:45.884419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.884419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.884719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] PATH_INFO: `/v3/projects/0c82ee17361740c9b919501bce700183/users/2dd31acd837b42708ab2bd050d5b550c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.887538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c82ee17361740c9b919501bce700183, user_id=2dd31acd837b42708ab2bd050d5b550c, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:45.890869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:45.904275 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1369/2751] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:45 2022] PUT /identity/v3/projects/0c82ee17361740c9b919501bce700183/users/2dd31acd837b42708ab2bd050d5b550c/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:08:45.916119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a32117-2586-4e20-a6a4-5522a5ccd875 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:45.917268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a32117-2586-4e20-a6a4-5522a5ccd875 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:45.917551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a32117-2586-4e20-a6a4-5522a5ccd875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:45.947032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-65a32117-2586-4e20-a6a4-5522a5ccd875 None None] MFA Rules not processed for user `2dd31acd837b42708ab2bd050d5b550c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:45.948856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-65a32117-2586-4e20-a6a4-5522a5ccd875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:46.040627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1383/2752] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:46.056091 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:46.058343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:46.117683 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:46.130004 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a84cd2-be68-4bfd-9bb3-8c24a4b24a55 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:46.135210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:46.135836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:46.136447 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:46.136726 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:46.146039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:46.147453 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:46.156909 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1370/2753] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:46.235452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a32117-2586-4e20-a6a4-5522a5ccd875 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:46.246104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a32117-2586-4e20-a6a4-5522a5ccd875 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:46.255300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e575047-a0c2-41b4-9fdd-cf8584bb47db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:46.255979 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e575047-a0c2-41b4-9fdd-cf8584bb47db service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:46.256288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e575047-a0c2-41b4-9fdd-cf8584bb47db service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:46.256704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e575047-a0c2-41b4-9fdd-cf8584bb47db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:46.305617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:46.324826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e575047-a0c2-41b4-9fdd-cf8584bb47db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:46.326279 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e575047-a0c2-41b4-9fdd-cf8584bb47db service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:46.337517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1384/2754] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:47.199542 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:47.201040 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:47.270099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43580b-5243-4457-90e9-53c6552d5333 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:47.275177 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:47.275799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:47.276075 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:47.276375 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] PATH_INFO: `/v3/users/12413144eaf5425cb29367dbec1ee189` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:47.277505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12413144eaf5425cb29367dbec1ee189)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:47.279388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:47.325261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12413144eaf5425cb29367dbec1ee189'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.337627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12413144eaf5425cb29367dbec1ee189'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.344192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12413144eaf5425cb29367dbec1ee189'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.344874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12413144eaf5425cb29367dbec1ee189'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.345468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12413144eaf5425cb29367dbec1ee189'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.357571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1371/2755] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:47 2022] DELETE /identity/v3/users/12413144eaf5425cb29367dbec1ee189 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:47.362024 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:47.362539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:47.362539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:47.380551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:47.381712 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:47.438029 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1385/2756] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:47.448188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:47.449615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:47.506659 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c4a6c-1a0f-4565-bd08-977783491392 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:47.510945 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:47.511807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:47.512084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:47.512388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:47.521617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:47.523606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:47.533553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1372/2757] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:47.821593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:47.824164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:47.839285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e44b66-5c22-471a-87d3-82210af848c4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:47.844450 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:47.845464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:47.845843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:47.846216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] PATH_INFO: `/v3/projects/e6ba55ea2518413e81eebb7df7477dca` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:47.847463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ba55ea2518413e81eebb7df7477dca)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:47.849573 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:47.905729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ba55ea2518413e81eebb7df7477dca', 'request_id': 'req-4cc19296-06f6-452d-a1ec-42958c7b6823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.912132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ba55ea2518413e81eebb7df7477dca', 'request_id': 'req-4cc19296-06f6-452d-a1ec-42958c7b6823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.918926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ba55ea2518413e81eebb7df7477dca', 'request_id': 'req-4cc19296-06f6-452d-a1ec-42958c7b6823'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.930670 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] The token cache is being invalidate because project e6ba55ea2518413e81eebb7df7477dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:47.930953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.931559 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:47.933162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1386/2758] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:47 2022] DELETE /identity/v3/projects/e6ba55ea2518413e81eebb7df7477dca => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:47.937201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:47.946971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa2aab-67c9-4e7a-8dc9-01d685c818c1 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:47.951480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:48.006575 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:48.007246 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:48.007536 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:48.007811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] PATH_INFO: `/v3/users/c8e37ad0de7448ad8d93e9433fc5699d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:48.008959 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e37ad0de7448ad8d93e9433fc5699d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:48.010917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:48.056471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e37ad0de7448ad8d93e9433fc5699d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.068051 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e37ad0de7448ad8d93e9433fc5699d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.075326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e37ad0de7448ad8d93e9433fc5699d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.075981 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e37ad0de7448ad8d93e9433fc5699d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.076612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e37ad0de7448ad8d93e9433fc5699d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.089531 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1373/2759] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:47 2022] DELETE /identity/v3/users/c8e37ad0de7448ad8d93e9433fc5699d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:48.364009 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:48.365698 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:48.378791 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc19296-06f6-452d-a1ec-42958c7b6823 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:48.435435 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:48.436017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:48.436330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:48.436610 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] PATH_INFO: `/v3/projects/0a049f222f0740a79e367a1ecf2b9a93` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:48.437719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a049f222f0740a79e367a1ecf2b9a93)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:48.439624 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:48.446146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:48.486745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a049f222f0740a79e367a1ecf2b9a93', 'request_id': 'req-6671ba57-d610-49c7-ade1-dde5fa545e97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.493689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a049f222f0740a79e367a1ecf2b9a93', 'request_id': 'req-6671ba57-d610-49c7-ade1-dde5fa545e97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.499236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a049f222f0740a79e367a1ecf2b9a93', 'request_id': 'req-6671ba57-d610-49c7-ade1-dde5fa545e97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.512038 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] The token cache is being invalidate because project 0a049f222f0740a79e367a1ecf2b9a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:48.512363 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.512967 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.514406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1387/2760] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:48 2022] DELETE /identity/v3/projects/0a049f222f0740a79e367a1ecf2b9a93 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:48.518680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:48.520110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:48.533016 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ac4c2-cf20-4e65-ac58-4984f5a91f26 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:48.588866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:48.589476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:48.589755 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:48.590025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] PATH_INFO: `/v3/users/744df4dd94b945b1b3333e919f5a785d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:48.591138 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=744df4dd94b945b1b3333e919f5a785d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:48.593070 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:48.626161 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:48.642088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '744df4dd94b945b1b3333e919f5a785d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.654630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '744df4dd94b945b1b3333e919f5a785d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.661830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '744df4dd94b945b1b3333e919f5a785d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.662515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '744df4dd94b945b1b3333e919f5a785d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.663124 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2641fed7-6f0b-4e07-b3a8-67a57a92ce72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '744df4dd94b945b1b3333e919f5a785d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:48.675479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1374/2761] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:08:48 2022] DELETE /identity/v3/users/744df4dd94b945b1b3333e919f5a785d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:48.967889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:48.969723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:48.984428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ba57-d610-49c7-ade1-dde5fa545e97 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.048265 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.048913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.049193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.049466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] PATH_INFO: `/v3/projects/236b140acc384a46b7c23782981ba5db` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.050581 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=236b140acc384a46b7c23782981ba5db)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.052509 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.103437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '236b140acc384a46b7c23782981ba5db', 'request_id': 'req-41bf72ae-a5f5-4a53-b16b-dc0284204f85'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:49.110095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '236b140acc384a46b7c23782981ba5db', 'request_id': 'req-41bf72ae-a5f5-4a53-b16b-dc0284204f85'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:49.117150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '236b140acc384a46b7c23782981ba5db', 'request_id': 'req-41bf72ae-a5f5-4a53-b16b-dc0284204f85'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:49.131001 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] The token cache is being invalidate because project 236b140acc384a46b7c23782981ba5db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:08:49.131285 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:49.131907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-41bf72ae-a5f5-4a53-b16b-dc0284204f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:08:49.133897 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1388/2762] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:08:48 2022] DELETE /identity/v3/projects/236b140acc384a46b7c23782981ba5db => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:49.140360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819002f5-bed6-409e-b758-86355ec0bcef None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.140645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819002f5-bed6-409e-b758-86355ec0bcef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.140917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819002f5-bed6-409e-b758-86355ec0bcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.160619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-819002f5-bed6-409e-b758-86355ec0bcef None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:49.161809 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-819002f5-bed6-409e-b758-86355ec0bcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:49.223080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1375/2763] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:49.231563 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f64764-aca1-439e-88cd-fa0123204eaf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.231856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f64764-aca1-439e-88cd-fa0123204eaf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.232437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f64764-aca1-439e-88cd-fa0123204eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.253857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c8f64764-aca1-439e-88cd-fa0123204eaf None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:49.255066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c8f64764-aca1-439e-88cd-fa0123204eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:49.264846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1389/2764] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:49.269271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819002f5-bed6-409e-b758-86355ec0bcef None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:49.279081 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819002f5-bed6-409e-b758-86355ec0bcef None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.286471 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.287089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.287418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.287816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.289838 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:08:49.290227 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.292662 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.308281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1376/2765] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:08:49.310656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f64764-aca1-439e-88cd-fa0123204eaf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:49.321634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f64764-aca1-439e-88cd-fa0123204eaf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.327776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.328609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.328887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.329170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.329917 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.331881 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.348545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:49.362759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1390/2766] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:08:49 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:49.367578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:49.383283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e263a-d427-4059-a8cf-64e657fbf4e3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.443366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.443986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.444291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.444590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.445916 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.447819 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.471734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1377/2767] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:08:49 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:08:49.475961 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:49.486334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4af168-5e2e-452d-9edb-8b89946a4e19 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.491244 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.492437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.492714 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.492985 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.493629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.495208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.505053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1391/2768] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:49.509821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:49.520678 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda8e1-f38c-4ce8-a3a8-d5ae5e5e4aba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.526009 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eb913-5204-4989-be27-44e89d59f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.526608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eb913-5204-4989-be27-44e89d59f458 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.526884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eb913-5204-4989-be27-44e89d59f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.527178 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eb913-5204-4989-be27-44e89d59f458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.527816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eb913-5204-4989-be27-44e89d59f458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.529408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eb913-5204-4989-be27-44e89d59f458 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.543979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1378/2769] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:08:49.545181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:49.555907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e89568-3ed0-46e8-865b-c48cf8192e8f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.560872 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.561558 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.561836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.562113 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] PATH_INFO: `/v3/projects/8412124c143b4c0987b078426f7fc9ce/users/ef37e092ffcf40b8b00a75c900f554b4/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.580167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8412124c143b4c0987b078426f7fc9ce, user_id=ef37e092ffcf40b8b00a75c900f554b4, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.582852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.595303 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1392/2770] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:08:49 2022] PUT /identity/v3/projects/8412124c143b4c0987b078426f7fc9ce/users/ef37e092ffcf40b8b00a75c900f554b4/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:08:49.666106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.666383 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.666653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.691969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:49.695895 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] MFA Rules not processed for user `ef37e092ffcf40b8b00a75c900f554b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:08:49.698137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:49.760022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1379/2771] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:08:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:08:49.771741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:08:49.773479 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:08:49.830738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf9929-4cbd-46d9-b280-485dd325a781 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:08:49.834346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d64d4-c357-4ab5-9edc-58779f77cd80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:08:49.834936 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d64d4-c357-4ab5-9edc-58779f77cd80 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:08:49.835211 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d64d4-c357-4ab5-9edc-58779f77cd80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:08:49.835480 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d64d4-c357-4ab5-9edc-58779f77cd80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:08:49.843409 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d64d4-c357-4ab5-9edc-58779f77cd80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:08:49.844861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d64d4-c357-4ab5-9edc-58779f77cd80 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:08:49.853857 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1393/2772] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:08:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:08:59.757104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=26 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:08:59.849458 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:07.343013 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:07.344706 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:07.410454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a98a1-e0ab-4066-8fe7-792dab0dfde5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:07.416734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:07.417371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:07.417652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:07.418022 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] PATH_INFO: `/v3/users/48dae07e9c5e4e73827a49fd47553955` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:07.419071 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] RBAC: Authorizing `identity:delete_user(user_id=48dae07e9c5e4e73827a49fd47553955)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:07.421392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:07.472508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48dae07e9c5e4e73827a49fd47553955'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:07.492538 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48dae07e9c5e4e73827a49fd47553955'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:07.493840 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48dae07e9c5e4e73827a49fd47553955'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:07.494673 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48dae07e9c5e4e73827a49fd47553955'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:07.495324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48dae07e9c5e4e73827a49fd47553955'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:07.509354 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1380/2773] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:07 2022] DELETE /identity/v3/users/48dae07e9c5e4e73827a49fd47553955 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:07.514046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:07.514455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:07.514455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:07.537861 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:07.539152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:07.607828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1394/2774] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:07.618128 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:07.619550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:07.675624 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdfcbd-faed-4403-b9c0-922025b0667b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:07.679133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:07.680384 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:07.680660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:07.680930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:07.689648 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:07.691057 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:07.700320 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1381/2775] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:07.959233 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:07.961289 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:07.973969 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d5065-6a48-4c27-9048-04e5756ffe97 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:07.979234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:07.979875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:07.980148 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:07.980452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] PATH_INFO: `/v3/projects/f9b6aff01dcb4dcfb97bbc4a8b7950c0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:07.981584 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b6aff01dcb4dcfb97bbc4a8b7950c0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:07.983500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:08.029336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b6aff01dcb4dcfb97bbc4a8b7950c0', 'request_id': 'req-3430b9be-9f80-46a0-bb3e-c71cf5681180'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.035934 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b6aff01dcb4dcfb97bbc4a8b7950c0', 'request_id': 'req-3430b9be-9f80-46a0-bb3e-c71cf5681180'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.042160 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b6aff01dcb4dcfb97bbc4a8b7950c0', 'request_id': 'req-3430b9be-9f80-46a0-bb3e-c71cf5681180'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.055890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] The token cache is being invalidate because project f9b6aff01dcb4dcfb97bbc4a8b7950c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:08.056196 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.056887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.058368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1395/2776] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:07 2022] DELETE /identity/v3/projects/f9b6aff01dcb4dcfb97bbc4a8b7950c0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:08.062388 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:08.071961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23977de0-c49f-4861-8dc3-86fad6236346 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:08.089776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:08.130426 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:08.131079 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:08.131361 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:08.131760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] PATH_INFO: `/v3/users/2dd31acd837b42708ab2bd050d5b550c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:08.132937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd31acd837b42708ab2bd050d5b550c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:08.135490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:08.305862 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd31acd837b42708ab2bd050d5b550c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.318077 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd31acd837b42708ab2bd050d5b550c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.325414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd31acd837b42708ab2bd050d5b550c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.326165 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd31acd837b42708ab2bd050d5b550c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.327055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-732c5905-62ee-424d-8a8c-4ead5446f505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd31acd837b42708ab2bd050d5b550c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.340856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1382/2777] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:08 2022] DELETE /identity/v3/users/2dd31acd837b42708ab2bd050d5b550c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:08.610400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:08.611911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:08.626981 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430b9be-9f80-46a0-bb3e-c71cf5681180 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:08.686236 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:08.686864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:08.687142 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:08.687455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] PATH_INFO: `/v3/projects/0c82ee17361740c9b919501bce700183` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:08.688593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c82ee17361740c9b919501bce700183)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:08.690644 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:08.722606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:08.742750 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c82ee17361740c9b919501bce700183', 'request_id': 'req-b11e0669-ced7-44a6-ac4a-5c31410afa42'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.749703 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c82ee17361740c9b919501bce700183', 'request_id': 'req-b11e0669-ced7-44a6-ac4a-5c31410afa42'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.756119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c82ee17361740c9b919501bce700183', 'request_id': 'req-b11e0669-ced7-44a6-ac4a-5c31410afa42'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.773054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] The token cache is being invalidate because project 0c82ee17361740c9b919501bce700183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:08.773361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.774530 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b11e0669-ced7-44a6-ac4a-5c31410afa42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:08.776013 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1396/2778] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:08 2022] DELETE /identity/v3/projects/0c82ee17361740c9b919501bce700183 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:08.780972 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab00d2-8375-4ae1-b944-01712c05529f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:08.781440 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab00d2-8375-4ae1-b944-01712c05529f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:08.781559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab00d2-8375-4ae1-b944-01712c05529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:08.801725 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f6ab00d2-8375-4ae1-b944-01712c05529f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:08.802984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f6ab00d2-8375-4ae1-b944-01712c05529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:08.860704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1383/2779] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:08.869207 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1a657-dae3-4d31-acf7-d0155c8a5df8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:08.869489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1a657-dae3-4d31-acf7-d0155c8a5df8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:08.869760 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1a657-dae3-4d31-acf7-d0155c8a5df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:08.890437 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bfc1a657-dae3-4d31-acf7-d0155c8a5df8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:08.891634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bfc1a657-dae3-4d31-acf7-d0155c8a5df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:08.899504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1397/2780] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:08.903739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab00d2-8375-4ae1-b944-01712c05529f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:08.913439 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab00d2-8375-4ae1-b944-01712c05529f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:08.918121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:08.918717 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:08.918996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:08.919277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:08.919979 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:09:08.920311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:08.921892 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:08.933561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1384/2781] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:09:08.937606 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1a657-dae3-4d31-acf7-d0155c8a5df8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:08.947272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1a657-dae3-4d31-acf7-d0155c8a5df8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:08.951840 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:08.952445 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:08.952748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:08.953020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:08.953642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:08.955517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:08.987334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1398/2782] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:08 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:08.991503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.002060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73599a-869e-4509-b1e9-00e29d23ce1d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.027885 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:09.058612 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.059134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.059409 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.059679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.060324 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.062199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.082986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1385/2783] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:08 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:09.087121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.097164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d87e02-3041-4c9d-98a1-0f4e8b19d0ee admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.102205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.102795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.103065 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.103330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.103963 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.105576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.115037 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1399/2784] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:09.119476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.129555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33732f-86dc-445d-bddb-65f87eae6dd5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.134204 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da573498-b5c9-402d-a529-ebfb59fa5402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.134791 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da573498-b5c9-402d-a529-ebfb59fa5402 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.135068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da573498-b5c9-402d-a529-ebfb59fa5402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.135339 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da573498-b5c9-402d-a529-ebfb59fa5402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.135973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da573498-b5c9-402d-a529-ebfb59fa5402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.137833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da573498-b5c9-402d-a529-ebfb59fa5402 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.147759 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1386/2785] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:09.151787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.161443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284c43-15c6-4540-9483-b9e6e0915898 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.166490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.167121 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.167397 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.167671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] PATH_INFO: `/v3/projects/1cc8c82cef8c401fb4afbb945d1fe330/users/f1b214a2d149461da3c39ee1485eb5df/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.182248 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc8c82cef8c401fb4afbb945d1fe330, user_id=f1b214a2d149461da3c39ee1485eb5df, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.184916 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.197790 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1400/2786] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:09 2022] PUT /identity/v3/projects/1cc8c82cef8c401fb4afbb945d1fe330/users/f1b214a2d149461da3c39ee1485eb5df/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:09.209050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209a06d-506f-43bd-a225-cea0f5c311cd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.209331 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209a06d-506f-43bd-a225-cea0f5c311cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.209599 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209a06d-506f-43bd-a225-cea0f5c311cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.236461 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0209a06d-506f-43bd-a225-cea0f5c311cd None None] MFA Rules not processed for user `f1b214a2d149461da3c39ee1485eb5df`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:09.237804 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0209a06d-506f-43bd-a225-cea0f5c311cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:09.298947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1387/2787] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:09.303423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.362555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe373-8866-425c-8098-69ea65400401 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.367034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.368173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.368478 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.368749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.369414 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.371282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.396124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1401/2788] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:09 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:09.400197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209a06d-506f-43bd-a225-cea0f5c311cd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.410637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209a06d-506f-43bd-a225-cea0f5c311cd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.466674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.466674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.466674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.466674 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.467326 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.469261 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.490385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1388/2789] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:09 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:09.494430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.504620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25378915-c38b-4a6d-8039-c0f0b5d9279a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.509162 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.509744 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.510019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.510287 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.510914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.512796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.522795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1402/2790] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:09.527199 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.537020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc61380-8c39-44dd-9d3a-9e23f166e1da admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.542476 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.543155 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.543443 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.543723 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] PATH_INFO: `/v3/projects/a29db13d34f64bbcaf0a085874b97762/users/b049991f5a0a4fe9ab15f24f6c5c38e4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.558145 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29db13d34f64bbcaf0a085874b97762, user_id=b049991f5a0a4fe9ab15f24f6c5c38e4, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.560798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.572902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1389/2791] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:09 2022] PUT /identity/v3/projects/a29db13d34f64bbcaf0a085874b97762/users/b049991f5a0a4fe9ab15f24f6c5c38e4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:09.576902 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.587183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8a226-0650-43c7-bc50-5e930c5482d5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.607477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:09.645055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666d46e-9922-44fb-96e0-6410cfa19d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.645699 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666d46e-9922-44fb-96e0-6410cfa19d43 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.645832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666d46e-9922-44fb-96e0-6410cfa19d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.646095 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666d46e-9922-44fb-96e0-6410cfa19d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.646733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666d46e-9922-44fb-96e0-6410cfa19d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.648962 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666d46e-9922-44fb-96e0-6410cfa19d43 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.658406 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1403/2792] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:09.662622 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.668641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:09.674919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cad65-5696-49b2-9e75-545675ad9894 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.679632 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.680317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.680597 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.680898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] PATH_INFO: `/v3/projects/a29db13d34f64bbcaf0a085874b97762/users/b049991f5a0a4fe9ab15f24f6c5c38e4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.682788 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29db13d34f64bbcaf0a085874b97762, user_id=b049991f5a0a4fe9ab15f24f6c5c38e4, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.685448 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.695284 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1390/2793] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:09 2022] PUT /identity/v3/projects/a29db13d34f64bbcaf0a085874b97762/users/b049991f5a0a4fe9ab15f24f6c5c38e4/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:09.705259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6155-76cd-4033-ba74-79df470c2e4c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.705543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6155-76cd-4033-ba74-79df470c2e4c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.705843 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6155-76cd-4033-ba74-79df470c2e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.733677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-16ca6155-76cd-4033-ba74-79df470c2e4c None None] MFA Rules not processed for user `b049991f5a0a4fe9ab15f24f6c5c38e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:09.734868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-16ca6155-76cd-4033-ba74-79df470c2e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:09.800909 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1404/2794] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:09.812088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.813884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:09.869987 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec875794-49a4-406a-9802-9220b4953ae7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.873686 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6ae69-d1be-4bd0-92d9-d5b94ce4a79d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.874286 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6ae69-d1be-4bd0-92d9-d5b94ce4a79d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.874562 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6ae69-d1be-4bd0-92d9-d5b94ce4a79d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.874833 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6ae69-d1be-4bd0-92d9-d5b94ce4a79d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.884068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6ae69-d1be-4bd0-92d9-d5b94ce4a79d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.885510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6ae69-d1be-4bd0-92d9-d5b94ce4a79d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:09.894971 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1391/2795] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:09.927946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca6155-76cd-4033-ba74-79df470c2e4c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:09.987743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca6155-76cd-4033-ba74-79df470c2e4c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:09.992175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:09.992818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:09.993092 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:09.993360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] PATH_INFO: `/v3/users/f1b214a2d149461da3c39ee1485eb5df` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:09.994482 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b214a2d149461da3c39ee1485eb5df)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:09.996413 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:10.041216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b214a2d149461da3c39ee1485eb5df'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.042366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:10.052152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b214a2d149461da3c39ee1485eb5df'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.062336 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b214a2d149461da3c39ee1485eb5df'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.063022 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b214a2d149461da3c39ee1485eb5df'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.075858 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b214a2d149461da3c39ee1485eb5df'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.081745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1405/2796] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:09 2022] DELETE /identity/v3/users/f1b214a2d149461da3c39ee1485eb5df => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:10.086921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c148f-5790-45a9-932e-778e49093717 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:10.087201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c148f-5790-45a9-932e-778e49093717 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:10.087473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c148f-5790-45a9-932e-778e49093717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:10.105691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e27c148f-5790-45a9-932e-778e49093717 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:10.106883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e27c148f-5790-45a9-932e-778e49093717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:10.166818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1392/2797] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:10.176647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:10.178143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:10.234111 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e223-4278-4c9e-bada-6c1522b45696 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:10.237743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:10.238337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:10.238609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:10.238880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:10.248223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:10.249705 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:10.261280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1406/2798] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:10.532072 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c148f-5790-45a9-932e-778e49093717 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:10.533519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e27c148f-5790-45a9-932e-778e49093717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:10.546172 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c148f-5790-45a9-932e-778e49093717 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:10.550760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:10.551387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:10.551657 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:10.552028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] PATH_INFO: `/v3/projects/1cc8c82cef8c401fb4afbb945d1fe330` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:10.553107 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc8c82cef8c401fb4afbb945d1fe330)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:10.555039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:10.601195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc8c82cef8c401fb4afbb945d1fe330', 'request_id': 'req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.608096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc8c82cef8c401fb4afbb945d1fe330', 'request_id': 'req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.613681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc8c82cef8c401fb4afbb945d1fe330', 'request_id': 'req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.624278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] The token cache is being invalidate because project 1cc8c82cef8c401fb4afbb945d1fe330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:10.624568 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.625214 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.626649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1393/2799] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:10 2022] DELETE /identity/v3/projects/1cc8c82cef8c401fb4afbb945d1fe330 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:10.630930 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:10.641353 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24379bc-ed4f-4050-a354-ec6c14fedf7b service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:10.696083 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:10.696752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:10.696929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:10.697202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] PATH_INFO: `/v3/users/b049991f5a0a4fe9ab15f24f6c5c38e4` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:10.698307 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b049991f5a0a4fe9ab15f24f6c5c38e4)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:10.700223 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:10.744618 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b049991f5a0a4fe9ab15f24f6c5c38e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.751240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b049991f5a0a4fe9ab15f24f6c5c38e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.758845 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b049991f5a0a4fe9ab15f24f6c5c38e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.759449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b049991f5a0a4fe9ab15f24f6c5c38e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.764927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:10.773197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c8be518a-0699-419d-af0e-9a4ea52e830b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b049991f5a0a4fe9ab15f24f6c5c38e4'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:10.779666 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1407/2800] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:10 2022] DELETE /identity/v3/users/b049991f5a0a4fe9ab15f24f6c5c38e4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:11.042445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.043936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:11.057355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d6dc5-2e52-4bfa-ae93-8b1c6908a609 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.069371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:11.114522 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.115148 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.115427 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.115702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] PATH_INFO: `/v3/projects/a29db13d34f64bbcaf0a085874b97762` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.116839 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29db13d34f64bbcaf0a085874b97762)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.119217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.163668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29db13d34f64bbcaf0a085874b97762', 'request_id': 'req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:11.170271 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29db13d34f64bbcaf0a085874b97762', 'request_id': 'req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:11.175775 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29db13d34f64bbcaf0a085874b97762', 'request_id': 'req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:11.186696 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] The token cache is being invalidate because project a29db13d34f64bbcaf0a085874b97762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:11.186978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:11.187593 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0ed89bd2-00fa-439b-9f88-ec51cc7bd0eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:11.189065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1394/2801] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:11 2022] DELETE /identity/v3/projects/a29db13d34f64bbcaf0a085874b97762 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:11.193944 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57a2f-d831-4964-9ced-2be5ca620ed9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.194255 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57a2f-d831-4964-9ced-2be5ca620ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.194519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff57a2f-d831-4964-9ced-2be5ca620ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.215328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8ff57a2f-d831-4964-9ced-2be5ca620ed9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:11.216551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8ff57a2f-d831-4964-9ced-2be5ca620ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:11.275597 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1408/2802] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:11.284406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4a576-8d44-476f-8a09-6aa6aaecdda9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.284690 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4a576-8d44-476f-8a09-6aa6aaecdda9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.284967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4a576-8d44-476f-8a09-6aa6aaecdda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.305306 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-30f4a576-8d44-476f-8a09-6aa6aaecdda9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:11.306559 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30f4a576-8d44-476f-8a09-6aa6aaecdda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:11.314237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1395/2803] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:11.318464 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff57a2f-d831-4964-9ced-2be5ca620ed9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.328134 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff57a2f-d831-4964-9ced-2be5ca620ed9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.332607 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.333193 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.333465 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.333733 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.334372 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:09:11.334669 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.336267 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.347256 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1409/2804] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:09:11.351381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4a576-8d44-476f-8a09-6aa6aaecdda9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.361021 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4a576-8d44-476f-8a09-6aa6aaecdda9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.365596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.366186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.366466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.366736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.367912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.369817 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.395846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1396/2805] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:11 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:11.399852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.409803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa4987-7b0a-4a70-b6d7-677790819b9e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.466052 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.466647 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.466920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.467216 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.467844 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.469761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.491228 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1410/2806] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:11 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:11.496210 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.508984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9c939-e8c7-4aa5-8022-22198e9550dc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.513870 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.514470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.514750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.515025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.515661 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.517270 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.527028 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1397/2807] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:11.530889 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.541175 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c0357-3999-41c9-aa6c-49e574e93bb5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.545718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffee72-17ee-42f3-a60d-3d2cc0f9847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.546461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffee72-17ee-42f3-a60d-3d2cc0f9847b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.546588 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffee72-17ee-42f3-a60d-3d2cc0f9847b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.546823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffee72-17ee-42f3-a60d-3d2cc0f9847b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.547456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffee72-17ee-42f3-a60d-3d2cc0f9847b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.549055 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffee72-17ee-42f3-a60d-3d2cc0f9847b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.559828 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1411/2808] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:11.563855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.573642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5b25d-10bb-463d-a3cf-4151c6ae4584 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.578237 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.578872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.579154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.579434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] PATH_INFO: `/v3/projects/d300d75330b04eda8e7e928f89210a14/users/b6b4ac5910514d87b1635655aeba59cc/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.593480 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d300d75330b04eda8e7e928f89210a14, user_id=b6b4ac5910514d87b1635655aeba59cc, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.596078 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.609395 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1398/2809] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:11 2022] PUT /identity/v3/projects/d300d75330b04eda8e7e928f89210a14/users/b6b4ac5910514d87b1635655aeba59cc/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:11.620213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb356-c724-4dfb-9ecb-be170f8ad426 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.620525 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb356-c724-4dfb-9ecb-be170f8ad426 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.620795 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beb356-c724-4dfb-9ecb-be170f8ad426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.647758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-22beb356-c724-4dfb-9ecb-be170f8ad426 None None] MFA Rules not processed for user `b6b4ac5910514d87b1635655aeba59cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:11.649433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-22beb356-c724-4dfb-9ecb-be170f8ad426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:11.689046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:11.713054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1412/2810] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:11.717473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.759904 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:11.779630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91c731-076e-418a-9b09-f96e1260ee80 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.784104 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.784734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.785010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.785281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.785912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.787802 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.814465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1399/2811] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:11 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:11.818689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb356-c724-4dfb-9ecb-be170f8ad426 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.829191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beb356-c724-4dfb-9ecb-be170f8ad426 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.885522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.886150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.886704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.886991 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.887629 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.889695 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.910723 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1413/2812] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:11 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:11.915466 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.925587 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e945-ae2e-445c-90e8-82f6d8739088 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.930100 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.930687 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.930962 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.931231 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.931856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.933465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.946109 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1400/2813] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:09:11.950323 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.960547 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1358c-0cdc-44c3-8400-39acbc35e02c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.966341 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4b60b-13a5-48ad-a517-1fe2cb97025c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:11.966939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4b60b-13a5-48ad-a517-1fe2cb97025c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:11.967218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4b60b-13a5-48ad-a517-1fe2cb97025c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:11.967486 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4b60b-13a5-48ad-a517-1fe2cb97025c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:11.968124 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4b60b-13a5-48ad-a517-1fe2cb97025c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:11.969741 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4b60b-13a5-48ad-a517-1fe2cb97025c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:11.980297 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1414/2814] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:11.984571 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:11.994961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fc380-5f0c-4c6c-b5ca-aa7fc38ee623 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:11.999623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.000446 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.000576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.000836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] PATH_INFO: `/v3/projects/8eac384160c54667a8687a962221d9a0/users/f3e78ed9f5c04e14948fd7f45e375d52/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.015386 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eac384160c54667a8687a962221d9a0, user_id=f3e78ed9f5c04e14948fd7f45e375d52, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.018108 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.031132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1401/2815] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:11 2022] PUT /identity/v3/projects/8eac384160c54667a8687a962221d9a0/users/f3e78ed9f5c04e14948fd7f45e375d52/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:12.042417 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becda90-a8b3-4c08-b858-e9393268f77b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.042704 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becda90-a8b3-4c08-b858-e9393268f77b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.043161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becda90-a8b3-4c08-b858-e9393268f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.071104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2becda90-a8b3-4c08-b858-e9393268f77b None None] MFA Rules not processed for user `f3e78ed9f5c04e14948fd7f45e375d52`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:12.072324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2becda90-a8b3-4c08-b858-e9393268f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:12.143120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1415/2816] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:12.147805 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:12.206318 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60260-19f7-4e9c-bb46-40b2e075c63a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:12.210898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.211484 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.211760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.212029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.212680 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.214551 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.240213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1402/2817] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:12 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:12.244913 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becda90-a8b3-4c08-b858-e9393268f77b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:12.255490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becda90-a8b3-4c08-b858-e9393268f77b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:12.300972 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:12.311823 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.312470 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.312774 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.313042 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.313662 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.315611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.337554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1416/2818] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:12 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:12.341930 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:12.352048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b7a6-15df-4ca7-bb22-b62c1152ac07 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:12.356692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.357282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.357567 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.357836 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.358464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.360137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.370343 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1403/2819] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:12.374176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:12.384346 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6bb8-62ef-4836-84b4-cf30d37e80d2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:12.389146 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.389778 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.390054 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.390324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] PATH_INFO: `/v3/projects/082d5f4e04dd40aa8b655775a74f0d1b/users/3cdd7d5c51a84eea900c4539bf22493b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.404893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=082d5f4e04dd40aa8b655775a74f0d1b, user_id=3cdd7d5c51a84eea900c4539bf22493b, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.407511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.419642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1417/2820] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:12 2022] PUT /identity/v3/projects/082d5f4e04dd40aa8b655775a74f0d1b/users/3cdd7d5c51a84eea900c4539bf22493b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:12.423414 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:12.433709 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74141bf5-ec71-4ad3-ae57-22bbf5634e65 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:12.488539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0692c-9ec3-4f4f-b209-174be0c0fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.489129 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0692c-9ec3-4f4f-b209-174be0c0fde6 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.489405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0692c-9ec3-4f4f-b209-174be0c0fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.489672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0692c-9ec3-4f4f-b209-174be0c0fde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.490293 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0692c-9ec3-4f4f-b209-174be0c0fde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.491846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0692c-9ec3-4f4f-b209-174be0c0fde6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.501655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1404/2821] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:12.505634 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:12.516368 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4d65-887f-49da-b8a8-1722f4e68a22 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:12.520874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.521501 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.521776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.522047 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] PATH_INFO: `/v3/projects/082d5f4e04dd40aa8b655775a74f0d1b/users/3cdd7d5c51a84eea900c4539bf22493b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.523959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=082d5f4e04dd40aa8b655775a74f0d1b, user_id=3cdd7d5c51a84eea900c4539bf22493b, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.526585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.536321 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1418/2822] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:12 2022] PUT /identity/v3/projects/082d5f4e04dd40aa8b655775a74f0d1b/users/3cdd7d5c51a84eea900c4539bf22493b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:12.547035 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5436f-00f9-4ea8-a7f4-0f02eced1e30 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.547406 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5436f-00f9-4ea8-a7f4-0f02eced1e30 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.547782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5436f-00f9-4ea8-a7f4-0f02eced1e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.555392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:12.577363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-64f5436f-00f9-4ea8-a7f4-0f02eced1e30 None None] MFA Rules not processed for user `3cdd7d5c51a84eea900c4539bf22493b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:12.578558 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-64f5436f-00f9-4ea8-a7f4-0f02eced1e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:12.643347 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1405/2823] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:12.654189 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:12.656032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:12.713139 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2ce03-180f-42f3-8849-b53bbd82fae4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:12.716745 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:12.717342 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:12.717621 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:12.717895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:12.726659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:12.728093 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:12.737874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1419/2824] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:13.226638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5436f-00f9-4ea8-a7f4-0f02eced1e30 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:13.236608 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5436f-00f9-4ea8-a7f4-0f02eced1e30 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:13.240062 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:13.240689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:13.240964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:13.241233 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:13.296020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:13.297482 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:13.306278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1406/2825] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:17.315181 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:17.316579 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:17.327580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0155d2-d0ce-4c43-a5e6-d2c21f6fde2b service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:17.331274 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff762893-692c-4310-93d3-4ff6d3e83b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:17.331890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762893-692c-4310-93d3-4ff6d3e83b19 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:17.332166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762893-692c-4310-93d3-4ff6d3e83b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:17.332469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762893-692c-4310-93d3-4ff6d3e83b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:17.387901 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff762893-692c-4310-93d3-4ff6d3e83b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:17.389491 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff762893-692c-4310-93d3-4ff6d3e83b19 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:17.398238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1420/2826] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:17.805265 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:17.806844 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:17.871708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b6792-d503-4611-b6ad-0244408b9592 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:17.876226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:17.876872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:17.877154 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:17.877431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] PATH_INFO: `/v3/users/ef37e092ffcf40b8b00a75c900f554b4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:17.878561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef37e092ffcf40b8b00a75c900f554b4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:17.880487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:17.910285 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:17.930346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef37e092ffcf40b8b00a75c900f554b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:17.942418 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef37e092ffcf40b8b00a75c900f554b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:17.949351 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef37e092ffcf40b8b00a75c900f554b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:17.950045 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef37e092ffcf40b8b00a75c900f554b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:17.950835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef37e092ffcf40b8b00a75c900f554b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:17.964179 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1407/2827] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:17 2022] DELETE /identity/v3/users/ef37e092ffcf40b8b00a75c900f554b4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:17.969205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:17.969585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:17.969879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:17.990812 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:17.991999 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:18.038511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:18.059053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1421/2828] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:18.068796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.070185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:18.125308 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98417f1-4a66-45d1-aa78-ef64f13e70e4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.128785 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9799db7-6072-41b4-a3ad-1d47dc3e6841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.129376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799db7-6072-41b4-a3ad-1d47dc3e6841 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.129649 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799db7-6072-41b4-a3ad-1d47dc3e6841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.129917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799db7-6072-41b4-a3ad-1d47dc3e6841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.140047 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9799db7-6072-41b4-a3ad-1d47dc3e6841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.141492 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9799db7-6072-41b4-a3ad-1d47dc3e6841 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.151822 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1408/2829] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:09:18.423906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.425367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:18.437856 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e7787-84f0-445d-9a35-18359eebaa4f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.442384 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.443011 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.443283 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.443572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] PATH_INFO: `/v3/projects/8412124c143b4c0987b078426f7fc9ce` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.444749 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8412124c143b4c0987b078426f7fc9ce)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.447459 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.493517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8412124c143b4c0987b078426f7fc9ce', 'request_id': 'req-30fb4453-cd14-4178-9f67-689e8e08fba0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:18.499636 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8412124c143b4c0987b078426f7fc9ce', 'request_id': 'req-30fb4453-cd14-4178-9f67-689e8e08fba0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:18.505319 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8412124c143b4c0987b078426f7fc9ce', 'request_id': 'req-30fb4453-cd14-4178-9f67-689e8e08fba0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:18.518120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] The token cache is being invalidate because project 8412124c143b4c0987b078426f7fc9ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:18.518430 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:18.519051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-30fb4453-cd14-4178-9f67-689e8e08fba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:18.520540 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1422/2830] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:18 2022] DELETE /identity/v3/projects/8412124c143b4c0987b078426f7fc9ce => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:18.525402 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25708d1-2667-4469-8c30-dcad8d74cde9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.525689 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25708d1-2667-4469-8c30-dcad8d74cde9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.525967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25708d1-2667-4469-8c30-dcad8d74cde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.544951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b25708d1-2667-4469-8c30-dcad8d74cde9 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:18.546134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b25708d1-2667-4469-8c30-dcad8d74cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:18.607570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1409/2831] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:18.615262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099bf35-ba3e-4347-b352-8d46dfd4cb66 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.615551 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099bf35-ba3e-4347-b352-8d46dfd4cb66 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.615818 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099bf35-ba3e-4347-b352-8d46dfd4cb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.636041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1099bf35-ba3e-4347-b352-8d46dfd4cb66 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:18.637477 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1099bf35-ba3e-4347-b352-8d46dfd4cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:18.645316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1423/2832] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:18.649508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25708d1-2667-4469-8c30-dcad8d74cde9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.659130 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25708d1-2667-4469-8c30-dcad8d74cde9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.663778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.664428 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.664720 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.664996 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.665637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:09:18.665942 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.667519 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.678637 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1410/2833] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:09:18.683182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099bf35-ba3e-4347-b352-8d46dfd4cb66 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.692202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099bf35-ba3e-4347-b352-8d46dfd4cb66 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.697682 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.698277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.698580 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.698854 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.699489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.701423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.727545 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1424/2834] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:18 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:18.731745 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.741634 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c9921-882c-4c4a-8a98-7c588c930e1a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.774391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:18.799462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.800050 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.800358 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.800628 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.801254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.803121 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.824134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1411/2835] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:18 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:18.828431 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.838929 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647407f9-f90b-4bbc-803e-630d40e2c892 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.843432 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.844017 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.844429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.844702 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.845339 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.846920 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.857056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1425/2836] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:18.860982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.870734 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e8087-740b-4f7d-80ee-0092462bd02e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.875342 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06783954-292c-44ad-9a95-44f1188b894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.875929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06783954-292c-44ad-9a95-44f1188b894f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.876211 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06783954-292c-44ad-9a95-44f1188b894f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.876523 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06783954-292c-44ad-9a95-44f1188b894f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.877158 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06783954-292c-44ad-9a95-44f1188b894f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.878728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06783954-292c-44ad-9a95-44f1188b894f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.888664 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1412/2837] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:18.892495 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:18.902572 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381d058-0a4d-418e-8c1c-efff41a4c55e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:18.907188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:18.907838 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.908116 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.908423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] PATH_INFO: `/v3/projects/e88eb628ecac4720be85e1d7b1266cba/users/9511143f4e294967acb95b2c9d657afd/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.923158 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e88eb628ecac4720be85e1d7b1266cba, user_id=9511143f4e294967acb95b2c9d657afd, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:18.925805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:18.938724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1426/2838] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:18 2022] PUT /identity/v3/projects/e88eb628ecac4720be85e1d7b1266cba/users/9511143f4e294967acb95b2c9d657afd/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:18.950313 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396ab8a-4d00-4b8a-9cd2-204261d3b5ba None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:18.950603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396ab8a-4d00-4b8a-9cd2-204261d3b5ba None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:18.950882 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396ab8a-4d00-4b8a-9cd2-204261d3b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:18.980164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d396ab8a-4d00-4b8a-9cd2-204261d3b5ba None None] MFA Rules not processed for user `9511143f4e294967acb95b2c9d657afd`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:18.981407 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d396ab8a-4d00-4b8a-9cd2-204261d3b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:19.047042 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1413/2839] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:19.051677 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.112564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bc4f8-412b-42a5-a5ee-5d313740902c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.117195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.117794 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.118066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.118330 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.118952 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.120866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.130839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:19.158867 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1427/2840] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:19 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:19.162947 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396ab8a-4d00-4b8a-9cd2-204261d3b5ba None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.174010 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396ab8a-4d00-4b8a-9cd2-204261d3b5ba None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.230975 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.231580 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.231861 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.232132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.232902 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.234872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.256099 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1414/2841] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:19 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:19.261659 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.272087 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a6a-9a59-44ed-bf37-ece97424c79c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.277320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.277911 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.278187 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.278456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.279086 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.280716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.296955 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1428/2842] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:19.298220 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.309088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d190e8-4289-42cc-bf19-9d5c28c9b8f6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.313647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d00834-5e38-4e5d-b50a-926af28db753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.314244 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d00834-5e38-4e5d-b50a-926af28db753 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.314518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d00834-5e38-4e5d-b50a-926af28db753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.314786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d00834-5e38-4e5d-b50a-926af28db753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.315413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d00834-5e38-4e5d-b50a-926af28db753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.317133 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d00834-5e38-4e5d-b50a-926af28db753 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.328262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1415/2843] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:19.332393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.343173 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8f992-67a7-4ab8-ad32-e341dc018eda admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.347903 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.348585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.348863 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.349138 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] PATH_INFO: `/v3/projects/5d8cc491631e4f8ebae95047da50d949/users/91754381a68b4506b7938932f407622a/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.363866 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8cc491631e4f8ebae95047da50d949, user_id=91754381a68b4506b7938932f407622a, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.366710 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.379922 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1429/2844] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:19 2022] PUT /identity/v3/projects/5d8cc491631e4f8ebae95047da50d949/users/91754381a68b4506b7938932f407622a/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:19.391527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea4a34-6579-440f-bc24-3e9e9e3e8831 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.391810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea4a34-6579-440f-bc24-3e9e9e3e8831 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.392134 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea4a34-6579-440f-bc24-3e9e9e3e8831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.421419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a8ea4a34-6579-440f-bc24-3e9e9e3e8831 None None] MFA Rules not processed for user `91754381a68b4506b7938932f407622a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:19.422617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a8ea4a34-6579-440f-bc24-3e9e9e3e8831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:19.432390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:19.495067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1416/2845] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:19.498801 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.499303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea4a34-6579-440f-bc24-3e9e9e3e8831 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.500489 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:19.560041 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea4a34-6579-440f-bc24-3e9e9e3e8831 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.564048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578c6c7-6489-474e-b348-6735ddd860ce admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.565106 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.565713 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.565993 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.566295 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.566994 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.568860 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.569065 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.569473 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.569748 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.570019 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] PATH_INFO: `/v3/users/b6b4ac5910514d87b1635655aeba59cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.571717 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b4ac5910514d87b1635655aeba59cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.573627 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.596153 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1417/2846] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:19 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:19.600311 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.622058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b4ac5910514d87b1635655aeba59cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:19.629171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b4ac5910514d87b1635655aeba59cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:19.638070 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b4ac5910514d87b1635655aeba59cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:19.638880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b4ac5910514d87b1635655aeba59cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:19.662311 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9348a2d2-d428-4768-8943-f74a87d053bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b4ac5910514d87b1635655aeba59cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:19.668264 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1430/2847] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:19 2022] DELETE /identity/v3/users/b6b4ac5910514d87b1635655aeba59cc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:19.668424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d80a6-9c03-4e5a-8f5d-ae641089eef0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.672566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.672847 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.673123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.692078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:19.693300 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:19.712431 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:19.723511 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:19.728417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.729025 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.729412 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.729691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.730315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.732205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.754636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1418/2848] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:19 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:19.757990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1431/2849] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:19.758452 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.760730 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:19.768325 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.769735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:19.772835 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8587e-baa9-43c8-b06a-eb300309d520 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.777619 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.778205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.778479 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.778752 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.779381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.781007 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.790642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1419/2850] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:09:19.794613 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.803776 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9e83e-3e53-46ca-a9a8-d4050fd17e89 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.808796 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.809419 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.809694 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.809968 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] PATH_INFO: `/v3/projects/76265f4010e64be7adb8f28443d75f26/users/39fc040ab465425ca6e20b4dd408ca18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.824741 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76265f4010e64be7adb8f28443d75f26, user_id=39fc040ab465425ca6e20b4dd408ca18, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.827074 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ae94f-3f2c-43ee-8104-302399a22d62 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.827433 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.830796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.831391 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.831664 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.831933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.840798 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1420/2851] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:19 2022] PUT /identity/v3/projects/76265f4010e64be7adb8f28443d75f26/users/39fc040ab465425ca6e20b4dd408ca18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:19.844539 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.844683 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.846183 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.856547 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1432/2852] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:09:19.909031 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677680d-8430-4106-9684-426ef0336929 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.914217 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b056e52-aa57-4ca8-97f8-0ab34eb957c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.914766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b056e52-aa57-4ca8-97f8-0ab34eb957c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.915647 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b056e52-aa57-4ca8-97f8-0ab34eb957c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.916023 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b056e52-aa57-4ca8-97f8-0ab34eb957c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.916859 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b056e52-aa57-4ca8-97f8-0ab34eb957c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.918722 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b056e52-aa57-4ca8-97f8-0ab34eb957c3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.932640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1421/2853] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:09:19.933817 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:19.943455 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb221968-ea20-49ee-be96-4831bf9fb9f7 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:19.949213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:19.949864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.950132 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.950402 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] PATH_INFO: `/v3/projects/76265f4010e64be7adb8f28443d75f26/users/39fc040ab465425ca6e20b4dd408ca18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:19.952475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=76265f4010e64be7adb8f28443d75f26, user_id=39fc040ab465425ca6e20b4dd408ca18, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:19.955056 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:19.967288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1433/2854] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:19 2022] PUT /identity/v3/projects/76265f4010e64be7adb8f28443d75f26/users/39fc040ab465425ca6e20b4dd408ca18/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:19.978845 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:19.979215 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:19.979565 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:20.009957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] MFA Rules not processed for user `39fc040ab465425ca6e20b4dd408ca18`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:20.011949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:20.082652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1422/2855] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:20.094534 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:20.096262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:20.123679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:20.125315 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:20.157408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:20.169028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da07333-0933-4946-95c8-54fa359418cd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:20.172758 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:20.173443 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:20.173787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:20.174153 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:20.175983 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:20.205363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe4ace-352e-4b2b-be2c-973a55aaf1b5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:20.210604 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:20.211256 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:20.211530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:20.212402 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] PATH_INFO: `/v3/projects/d300d75330b04eda8e7e928f89210a14` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:20.213641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d300d75330b04eda8e7e928f89210a14)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:20.215550 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:20.235966 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:20.237485 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:20.247377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1434/2856] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:20.263151 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd300d75330b04eda8e7e928f89210a14', 'request_id': 'req-5aebc94f-2600-496e-badf-886911d6b9ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.271956 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd300d75330b04eda8e7e928f89210a14', 'request_id': 'req-5aebc94f-2600-496e-badf-886911d6b9ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.278758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd300d75330b04eda8e7e928f89210a14', 'request_id': 'req-5aebc94f-2600-496e-badf-886911d6b9ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.290430 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] The token cache is being invalidate because project d300d75330b04eda8e7e928f89210a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:20.290919 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.291780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.293465 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1423/2857] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:20 2022] DELETE /identity/v3/projects/d300d75330b04eda8e7e928f89210a14 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:20.297614 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:20.310240 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca90df-8d76-4785-8038-95a3b86985bb service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:20.323229 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:20.367813 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:20.368994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:20.369329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:20.369552 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] PATH_INFO: `/v3/users/f3e78ed9f5c04e14948fd7f45e375d52` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:20.370876 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e78ed9f5c04e14948fd7f45e375d52)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:20.372846 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:20.420797 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e78ed9f5c04e14948fd7f45e375d52'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.427875 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e78ed9f5c04e14948fd7f45e375d52'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.436000 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e78ed9f5c04e14948fd7f45e375d52'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.436667 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e78ed9f5c04e14948fd7f45e375d52'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.450218 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e78ed9f5c04e14948fd7f45e375d52'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.457208 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1435/2858] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:20 2022] DELETE /identity/v3/users/f3e78ed9f5c04e14948fd7f45e375d52 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:20.756912 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:20.759276 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:20.772216 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc94f-2600-496e-badf-886911d6b9ae admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:20.827931 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:20.828644 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:20.828921 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:20.829189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] PATH_INFO: `/v3/projects/8eac384160c54667a8687a962221d9a0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:20.830291 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eac384160c54667a8687a962221d9a0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:20.832150 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:20.875874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eac384160c54667a8687a962221d9a0', 'request_id': 'req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.878328 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:20.883316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eac384160c54667a8687a962221d9a0', 'request_id': 'req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.890731 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eac384160c54667a8687a962221d9a0', 'request_id': 'req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.901986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] The token cache is being invalidate because project 8eac384160c54667a8687a962221d9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:20.902272 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.902936 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:20.904415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1424/2859] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:20 2022] DELETE /identity/v3/projects/8eac384160c54667a8687a962221d9a0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:20.908555 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:20.909994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:20.922544 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeaa7c-2020-463b-8966-c6422ccbc6cf admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:20.978149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:20.978700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:20.978980 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:20.979253 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] PATH_INFO: `/v3/users/3cdd7d5c51a84eea900c4539bf22493b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:20.980395 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cdd7d5c51a84eea900c4539bf22493b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:20.982313 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.021352 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:21.028512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cdd7d5c51a84eea900c4539bf22493b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.035729 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cdd7d5c51a84eea900c4539bf22493b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.042803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cdd7d5c51a84eea900c4539bf22493b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.043421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cdd7d5c51a84eea900c4539bf22493b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.055502 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fc68f22-5be2-47de-82ef-f425f4b2f853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cdd7d5c51a84eea900c4539bf22493b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.061280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1436/2860] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:20 2022] DELETE /identity/v3/users/3cdd7d5c51a84eea900c4539bf22493b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:21.321831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.323255 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:21.335923 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75554d-3ddc-4d00-a3c7-a45ef89f714f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.392205 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.392864 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.393143 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.393415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] PATH_INFO: `/v3/projects/082d5f4e04dd40aa8b655775a74f0d1b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.394520 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=082d5f4e04dd40aa8b655775a74f0d1b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.396478 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.443813 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082d5f4e04dd40aa8b655775a74f0d1b', 'request_id': 'req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.450769 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082d5f4e04dd40aa8b655775a74f0d1b', 'request_id': 'req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.456186 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082d5f4e04dd40aa8b655775a74f0d1b', 'request_id': 'req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.467360 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] The token cache is being invalidate because project 082d5f4e04dd40aa8b655775a74f0d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:21.467643 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.468296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1b3f0fbb-934c-4690-af89-4a870a7e3b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:21.469770 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1425/2861] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:21 2022] DELETE /identity/v3/projects/082d5f4e04dd40aa8b655775a74f0d1b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:21.475593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca38f8-e022-4401-a097-78201ba25839 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.475878 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca38f8-e022-4401-a097-78201ba25839 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.476152 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca38f8-e022-4401-a097-78201ba25839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.495933 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-95ca38f8-e022-4401-a097-78201ba25839 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:21.497195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-95ca38f8-e022-4401-a097-78201ba25839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:21.555048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1437/2862] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:21.563379 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291ce86-623f-4c89-abe2-1d490a353096 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.563668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291ce86-623f-4c89-abe2-1d490a353096 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.563944 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291ce86-623f-4c89-abe2-1d490a353096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.584518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e291ce86-623f-4c89-abe2-1d490a353096 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:21.585738 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e291ce86-623f-4c89-abe2-1d490a353096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:21.595269 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1426/2863] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:21.599396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca38f8-e022-4401-a097-78201ba25839 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.616436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca38f8-e022-4401-a097-78201ba25839 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.616436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.616436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.616436 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.616877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.616877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:09:21.616877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.617716 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.632280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1438/2864] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:09:21.634445 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291ce86-623f-4c89-abe2-1d490a353096 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.643779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291ce86-623f-4c89-abe2-1d490a353096 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.648518 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.649110 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.649868 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.650136 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.650758 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.652646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.680823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1427/2865] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:21 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:21.684853 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.694832 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb14f2-8a14-4ba4-a0e2-8832fb0ac988 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.758531 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.759143 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.759423 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.759700 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.760379 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.762536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.773487 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.776425 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:21.784433 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1439/2866] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:21 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:21.788357 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.798466 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55f5d-1166-4c24-a360-11cbc1bd165b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.803288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.803874 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.804150 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.804467 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.805119 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.806821 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.817310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1440/2867] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:09:21.821928 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.826001 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:21.831894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b053b9-b6e3-49cf-a0f2-4b478cddf4ac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.837366 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.837949 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.838224 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.838492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.839112 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.840585 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f009bd8-ff7f-4254-9da6-7eb7432796ef admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.841059 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.844373 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.844974 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.845254 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.845530 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.847278 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:21.850536 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1441/2868] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:21.854625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:21.864574 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351542cd-263e-4edf-80d3-935206b3e754 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:21.869349 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c5812-cc7e-49ae-b390-41b7499e0098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:21.870006 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c5812-cc7e-49ae-b390-41b7499e0098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.870288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c5812-cc7e-49ae-b390-41b7499e0098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.870548 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c5812-cc7e-49ae-b390-41b7499e0098 admin admin] PATH_INFO: `/v3/projects/d188ebd93f784d2c98648a16cd6294f5/users/c2e7e8504f84433aaf58a33b9a51e87a/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:21.878782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:21.886938 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c5812-cc7e-49ae-b390-41b7499e0098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d188ebd93f784d2c98648a16cd6294f5, user_id=c2e7e8504f84433aaf58a33b9a51e87a, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.890576 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c5812-cc7e-49ae-b390-41b7499e0098 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.905393 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1442/2869] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:21 2022] PUT /identity/v3/projects/d188ebd93f784d2c98648a16cd6294f5/users/c2e7e8504f84433aaf58a33b9a51e87a/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:21.920456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:21.920456 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:21.923015 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1428/2870] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:09:21.980656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece2e7f-5e01-4ea8-aba8-7f58e67396c9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:21.981246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece2e7f-5e01-4ea8-aba8-7f58e67396c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:21.981246 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece2e7f-5e01-4ea8-aba8-7f58e67396c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.014171 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dece2e7f-5e01-4ea8-aba8-7f58e67396c9 None None] MFA Rules not processed for user `c2e7e8504f84433aaf58a33b9a51e87a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:22.015632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dece2e7f-5e01-4ea8-aba8-7f58e67396c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:22.084295 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1443/2871] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:22.089424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.099846 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7cc5b-1fae-47ee-baea-873811a8b5fe service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.160029 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.160778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.161069 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.161312 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.161939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.163803 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.190314 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1429/2872] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:22 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:22.194492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece2e7f-5e01-4ea8-aba8-7f58e67396c9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.205656 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece2e7f-5e01-4ea8-aba8-7f58e67396c9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.266990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.267523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.267800 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.268068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.268759 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.270678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.295562 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1444/2873] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:22 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:22.300096 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.311251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33d2d6-330a-4186-aca6-0800d2abfc79 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.317094 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.317834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.318319 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.318704 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.319475 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.321719 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.332714 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1430/2874] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:22.337235 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.347663 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea620-5e03-470e-b822-f70f322bf587 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.352833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.354484 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.354761 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.355034 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] PATH_INFO: `/v3/projects/169b1711851a411185f6c2f47590ec3b/users/e49869791f63470fab4c13cef0e5c77b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.375066 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169b1711851a411185f6c2f47590ec3b, user_id=e49869791f63470fab4c13cef0e5c77b, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.377740 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.391428 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1445/2875] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:22 2022] PUT /identity/v3/projects/169b1711851a411185f6c2f47590ec3b/users/e49869791f63470fab4c13cef0e5c77b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:22.395405 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.405681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babac50f-dfa2-4250-9862-80912c9fc12a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.467316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.467855 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.468131 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.468434 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.469068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.470640 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.472195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.480869 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1431/2876] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:22.484867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.494672 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9729bac-53b3-4ad4-a993-ac6e8c356835 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.499251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c3ec9-2416-40ea-b83e-c3bb62b4100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.499867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3ec9-2416-40ea-b83e-c3bb62b4100a admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.500146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3ec9-2416-40ea-b83e-c3bb62b4100a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.500451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3ec9-2416-40ea-b83e-c3bb62b4100a admin admin] PATH_INFO: `/v3/projects/169b1711851a411185f6c2f47590ec3b/users/e49869791f63470fab4c13cef0e5c77b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.502322 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c3ec9-2416-40ea-b83e-c3bb62b4100a admin admin] RBAC: Authorizing `identity:create_grant(project_id=169b1711851a411185f6c2f47590ec3b, user_id=e49869791f63470fab4c13cef0e5c77b, role_id=78859ba2c4f645bea8c7e9d0fa00c5aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.505019 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c3ec9-2416-40ea-b83e-c3bb62b4100a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.507221 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:22.519705 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1432/2877] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:22 2022] PUT /identity/v3/projects/169b1711851a411185f6c2f47590ec3b/users/e49869791f63470fab4c13cef0e5c77b/roles/78859ba2c4f645bea8c7e9d0fa00c5aa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:22.520585 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:22.530304 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.530590 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.530866 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.532605 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e0162-b9cd-4e76-a0aa-9bfb37148c73 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.558548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] MFA Rules not processed for user `e49869791f63470fab4c13cef0e5c77b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:22.559760 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:22.583031 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.583617 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.583893 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.584164 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.585941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:22.629922 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1433/2878] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:22.641952 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.643503 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:22.651675 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.653247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.657174 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5fafe-e627-4dca-976b-9dc5a9d363f3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.661807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.662417 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.662691 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.662963 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.663361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1446/2879] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:09:22.673084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.674524 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.684463 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1434/2880] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:22 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:09:22.733852 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:22.742269 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc5c96-344e-4def-8909-6f64e96382cb service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:22.746600 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49f64c-8d4c-4a11-a443-5034ebea2f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:22.747361 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49f64c-8d4c-4a11-a443-5034ebea2f2b service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:22.747713 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49f64c-8d4c-4a11-a443-5034ebea2f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:22.748027 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49f64c-8d4c-4a11-a443-5034ebea2f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:22.804296 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49f64c-8d4c-4a11-a443-5034ebea2f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:22.806337 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49f64c-8d4c-4a11-a443-5034ebea2f2b service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:22.815041 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1447/2881] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:26.072851 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:26.132997 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b6321-2e61-42ea-b14e-00fcd392397a service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:26.137743 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:26.138359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.138641 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.138917 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] PATH_INFO: `/v3/users/c2e7e8504f84433aaf58a33b9a51e87a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.140043 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e7e8504f84433aaf58a33b9a51e87a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:26.142038 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:26.187084 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e7e8504f84433aaf58a33b9a51e87a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.198396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e7e8504f84433aaf58a33b9a51e87a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.200353 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:26.206729 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e7e8504f84433aaf58a33b9a51e87a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.207394 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e7e8504f84433aaf58a33b9a51e87a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.208011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e7e8504f84433aaf58a33b9a51e87a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.220757 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1435/2882] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:26 2022] DELETE /identity/v3/users/c2e7e8504f84433aaf58a33b9a51e87a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:26.225284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.225764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.225880 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.246941 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:26.248127 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.288946 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:26.307419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1448/2883] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:26.317163 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:26.318557 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.374701 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bc96-1c97-4841-bfd1-ea5dc7503dba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:26.378369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:26.378965 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.379243 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.379515 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.388553 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:26.389973 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:26.399296 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1436/2884] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:26.665492 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:26.667837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.681826 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112c43-5ea9-43f6-879f-921b3ca13ef4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:26.686814 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:26.687510 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.687870 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.688213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] PATH_INFO: `/v3/projects/d188ebd93f784d2c98648a16cd6294f5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.689708 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d188ebd93f784d2c98648a16cd6294f5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:26.691907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:26.706537 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:26.708376 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.722363 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c3e8b-6f9b-4d6e-b87c-05de9a2a1490 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:26.727504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:26.728132 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.728505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.728790 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] PATH_INFO: `/v3/users/9511143f4e294967acb95b2c9d657afd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.729995 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9511143f4e294967acb95b2c9d657afd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:26.731937 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:26.741523 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd188ebd93f784d2c98648a16cd6294f5', 'request_id': 'req-6868d188-3c26-4c4a-add1-f503e2b50a9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.747351 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd188ebd93f784d2c98648a16cd6294f5', 'request_id': 'req-6868d188-3c26-4c4a-add1-f503e2b50a9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.752877 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd188ebd93f784d2c98648a16cd6294f5', 'request_id': 'req-6868d188-3c26-4c4a-add1-f503e2b50a9f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.766020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] The token cache is being invalidate because project d188ebd93f784d2c98648a16cd6294f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:26.766334 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.766971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.768461 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1449/2885] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:26 2022] DELETE /identity/v3/projects/d188ebd93f784d2c98648a16cd6294f5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:26.772348 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:26.773718 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.779369 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9511143f4e294967acb95b2c9d657afd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.791630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9511143f4e294967acb95b2c9d657afd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.798365 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9511143f4e294967acb95b2c9d657afd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.799039 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9511143f4e294967acb95b2c9d657afd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.799642 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f8f9b8f2-0a68-4058-9f4b-248169393eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9511143f4e294967acb95b2c9d657afd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.812899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1437/2886] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:26 2022] DELETE /identity/v3/users/9511143f4e294967acb95b2c9d657afd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:26.817218 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.817576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.817881 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.836966 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:26.837483 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868d188-3c26-4c4a-add1-f503e2b50a9f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:26.838146 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.890323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:26.893615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:26.894238 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.894641 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.894925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] PATH_INFO: `/v3/users/e49869791f63470fab4c13cef0e5c77b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.896117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49869791f63470fab4c13cef0e5c77b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:26.898360 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:26.903630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1438/2887] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:26.910164 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:26.911527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.922756 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:26.950561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49869791f63470fab4c13cef0e5c77b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.958202 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49869791f63470fab4c13cef0e5c77b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.968165 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49869791f63470fab4c13cef0e5c77b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.969069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49869791f63470fab4c13cef0e5c77b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.976830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2288913-5caf-4ce5-8da4-d808812b1293 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:26.980540 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:26.981142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:26.981413 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:26.981681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:26.981850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49869791f63470fab4c13cef0e5c77b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:26.983736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:26.988021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1450/2888] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:26 2022] DELETE /identity/v3/users/e49869791f63470fab4c13cef0e5c77b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:26.996653 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:26.998011 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:27.007815 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1439/2889] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:09:27.435950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:27.437603 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:27.485344 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:27.486934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:27.501341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70870-5b32-4644-9ad0-effb4893e56c service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:27.517926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064351c-a019-4a0a-abba-bae9a39e7a46 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:27.522953 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:27.523681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.524046 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.524356 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] PATH_INFO: `/v3/projects/169b1711851a411185f6c2f47590ec3b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.525500 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] RBAC: Authorizing `identity:delete_project(project_id=169b1711851a411185f6c2f47590ec3b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:27.528120 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:27.562570 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:27.563226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.563504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.563778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] PATH_INFO: `/v3/projects/e88eb628ecac4720be85e1d7b1266cba` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.564982 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e88eb628ecac4720be85e1d7b1266cba)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:27.567061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:27.578973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169b1711851a411185f6c2f47590ec3b', 'request_id': 'req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.585401 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169b1711851a411185f6c2f47590ec3b', 'request_id': 'req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.591596 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169b1711851a411185f6c2f47590ec3b', 'request_id': 'req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.605632 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] The token cache is being invalidate because project 169b1711851a411185f6c2f47590ec3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:27.606002 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.606719 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-044a5d47-e1f5-4f6b-8282-3e3eeb35c32b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.608345 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1451/2890] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:27 2022] DELETE /identity/v3/projects/169b1711851a411185f6c2f47590ec3b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:27.613410 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b97399-2666-4ad4-8b07-4b482aef94f1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.613776 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b97399-2666-4ad4-8b07-4b482aef94f1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.614057 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b97399-2666-4ad4-8b07-4b482aef94f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.623030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e88eb628ecac4720be85e1d7b1266cba', 'request_id': 'req-94c16a01-449d-4dd9-9b07-e2171fbdce62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.631157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e88eb628ecac4720be85e1d7b1266cba', 'request_id': 'req-94c16a01-449d-4dd9-9b07-e2171fbdce62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.633997 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-45b97399-2666-4ad4-8b07-4b482aef94f1 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:27.635226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-45b97399-2666-4ad4-8b07-4b482aef94f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:27.637941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e88eb628ecac4720be85e1d7b1266cba', 'request_id': 'req-94c16a01-449d-4dd9-9b07-e2171fbdce62'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.648897 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] The token cache is being invalidate because project e88eb628ecac4720be85e1d7b1266cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:27.649185 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.649830 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.651371 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1440/2891] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:27 2022] DELETE /identity/v3/projects/e88eb628ecac4720be85e1d7b1266cba => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:27.655196 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:27.656419 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:27.656552 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:27.693282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:27.706791 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1452/2892] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:27.708268 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb645e-88e5-41f4-9894-d3e6642c3783 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.708625 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb645e-88e5-41f4-9894-d3e6642c3783 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.708906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb645e-88e5-41f4-9894-d3e6642c3783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.723504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c16a01-449d-4dd9-9b07-e2171fbdce62 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:27.728036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4dbb645e-88e5-41f4-9894-d3e6642c3783 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:27.728201 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:27.728742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.729510 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.729779 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] PATH_INFO: `/v3/users/91754381a68b4506b7938932f407622a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.729925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4dbb645e-88e5-41f4-9894-d3e6642c3783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:27.730926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91754381a68b4506b7938932f407622a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:27.732842 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:27.738028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1453/2893] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:27.742316 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb645e-88e5-41f4-9894-d3e6642c3783 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:27.751908 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb645e-88e5-41f4-9894-d3e6642c3783 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:27.757136 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:27.757767 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.758077 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.758369 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.759103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:09:27.759449 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:27.761170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:27.774021 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1454/2894] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:09:27.778767 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:27.782061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91754381a68b4506b7938932f407622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.788048 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dff35-e8d4-4ba8-a5f4-374aa592c273 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:27.793385 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:27.793957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.794234 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.794505 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.794949 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91754381a68b4506b7938932f407622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.795123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:27.797036 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:27.801629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91754381a68b4506b7938932f407622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.802307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91754381a68b4506b7938932f407622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.802929 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91754381a68b4506b7938932f407622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:27.816581 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1441/2895] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:27 2022] DELETE /identity/v3/users/91754381a68b4506b7938932f407622a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:27.828277 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1455/2896] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:27 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:27.832067 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:27.833610 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:27.901359 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f4eb9-b7ca-4a94-8e20-876b33466646 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:27.956596 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:27.957195 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:27.957470 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:27.957739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:27.958366 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:27.960274 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:27.970825 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:27.988289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1442/2897] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:27 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:27.992301 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.002781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e639-cfc5-4df4-8d1a-c0a1dba2b728 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.007488 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.008282 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.008567 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.009226 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.009891 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.011735 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.021781 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1456/2898] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 02:09:28.026213 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.036020 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cb280-9790-4578-a698-c8cc6b489b12 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.040627 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.041222 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.042112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.042390 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.043036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.044639 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.054586 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1443/2899] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:28.058752 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.068630 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd6444-1d85-4f43-afed-64c186cd129d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.073259 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36057dca-e944-4e8f-8e05-e64992a2f76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.073910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36057dca-e944-4e8f-8e05-e64992a2f76a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.074188 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36057dca-e944-4e8f-8e05-e64992a2f76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.074453 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36057dca-e944-4e8f-8e05-e64992a2f76a admin admin] PATH_INFO: `/v3/projects/eeef9069ad924b51a74d509cab103c0a/users/f38226532a0d4d4eb1cc3790a0ae7a97/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.090830 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36057dca-e944-4e8f-8e05-e64992a2f76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeef9069ad924b51a74d509cab103c0a, user_id=f38226532a0d4d4eb1cc3790a0ae7a97, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.093681 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36057dca-e944-4e8f-8e05-e64992a2f76a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.101219 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.103073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.110103 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1457/2900] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:28 2022] PUT /identity/v3/projects/eeef9069ad924b51a74d509cab103c0a/users/f38226532a0d4d4eb1cc3790a0ae7a97/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:28.122195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.122517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.122883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.127055 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5df7-551b-4676-846d-577e553440bf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.155104 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] MFA Rules not processed for user `f38226532a0d4d4eb1cc3790a0ae7a97`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:28.156743 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.185462 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.186089 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.186362 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.186633 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] PATH_INFO: `/v3/projects/5d8cc491631e4f8ebae95047da50d949` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.187843 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8cc491631e4f8ebae95047da50d949)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.189811 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.221519 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1458/2901] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:28.233161 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.234609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.241171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8cc491631e4f8ebae95047da50d949', 'request_id': 'req-852fa884-f6df-4404-914a-2900de349911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.249277 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8cc491631e4f8ebae95047da50d949', 'request_id': 'req-852fa884-f6df-4404-914a-2900de349911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.255666 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8cc491631e4f8ebae95047da50d949', 'request_id': 'req-852fa884-f6df-4404-914a-2900de349911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.267799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] The token cache is being invalidate because project 5d8cc491631e4f8ebae95047da50d949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:28.268086 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.268872 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.270387 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1444/2902] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:28 2022] DELETE /identity/v3/projects/5d8cc491631e4f8ebae95047da50d949 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:28.274157 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.275460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.280308 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:28.296504 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463edca-2f7f-48ac-9290-fcd365547381 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.343652 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fa884-f6df-4404-914a-2900de349911 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.346642 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.347244 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.347522 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.347796 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.348561 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.349184 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.349467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.349595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.349739 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] PATH_INFO: `/v3/users/39fc040ab465425ca6e20b4dd408ca18` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.350876 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=39fc040ab465425ca6e20b4dd408ca18)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.352823 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.363978 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:28.403918 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39fc040ab465425ca6e20b4dd408ca18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.409835 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.411343 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.415749 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39fc040ab465425ca6e20b4dd408ca18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.421026 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1459/2903] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 02:09:28.422992 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39fc040ab465425ca6e20b4dd408ca18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.423675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39fc040ab465425ca6e20b4dd408ca18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.424316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab28d85a-4f90-4a6a-8b28-4a1e65cc0d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39fc040ab465425ca6e20b4dd408ca18'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.439424 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1445/2904] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:28 2022] DELETE /identity/v3/users/39fc040ab465425ca6e20b4dd408ca18 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:28.440651 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.442324 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.456107 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6eb1-145c-42d5-993d-5908cc51307f service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.514650 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.515262 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.515538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.515808 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] PATH_INFO: `/v3/users/f38226532a0d4d4eb1cc3790a0ae7a97` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.517128 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38226532a0d4d4eb1cc3790a0ae7a97)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.519033 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.565293 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38226532a0d4d4eb1cc3790a0ae7a97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.569931 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:28.573742 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38226532a0d4d4eb1cc3790a0ae7a97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.586715 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38226532a0d4d4eb1cc3790a0ae7a97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.587556 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38226532a0d4d4eb1cc3790a0ae7a97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.602868 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38226532a0d4d4eb1cc3790a0ae7a97'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.611638 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1460/2905] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:28 2022] DELETE /identity/v3/users/f38226532a0d4d4eb1cc3790a0ae7a97 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:28.615171 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.615688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.615818 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.635323 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:28.637030 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.698415 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1446/2906] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:28.709615 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.711149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.725374 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:28.728289 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:28.745626 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83371e8-e5ef-435c-9cd8-568e5cea39a5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.751786 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.752491 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.752911 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.753189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] PATH_INFO: `/v3/projects/76265f4010e64be7adb8f28443d75f26` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.755142 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76265f4010e64be7adb8f28443d75f26)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.758229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.787950 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f0ddb-b665-4956-9598-f7906f36e393 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:28.792554 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:28.793184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:28.793462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:28.793737 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:28.802837 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:28.804288 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:28.806611 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76265f4010e64be7adb8f28443d75f26', 'request_id': 'req-81dd995c-a248-48cc-9b40-c34b00f4b035'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.814060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76265f4010e64be7adb8f28443d75f26', 'request_id': 'req-81dd995c-a248-48cc-9b40-c34b00f4b035'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.814517 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1461/2907] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:28.820027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76265f4010e64be7adb8f28443d75f26', 'request_id': 'req-81dd995c-a248-48cc-9b40-c34b00f4b035'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.831303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] The token cache is being invalidate because project 76265f4010e64be7adb8f28443d75f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:28.831592 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.832280 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81dd995c-a248-48cc-9b40-c34b00f4b035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:28.833777 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1447/2908] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:28 2022] DELETE /identity/v3/projects/76265f4010e64be7adb8f28443d75f26 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:29.113925 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.115376 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:29.128751 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54aceb-7d24-410c-a861-2f652cd0ae43 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.186251 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.186906 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.187186 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.187463 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] PATH_INFO: `/v3/projects/eeef9069ad924b51a74d509cab103c0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.188622 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeef9069ad924b51a74d509cab103c0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.190515 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.235167 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeef9069ad924b51a74d509cab103c0a', 'request_id': 'req-43ef1f60-2203-490a-9b68-63caa0f140f7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:29.241125 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeef9069ad924b51a74d509cab103c0a', 'request_id': 'req-43ef1f60-2203-490a-9b68-63caa0f140f7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:29.246620 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeef9069ad924b51a74d509cab103c0a', 'request_id': 'req-43ef1f60-2203-490a-9b68-63caa0f140f7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:29.258918 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] The token cache is being invalidate because project eeef9069ad924b51a74d509cab103c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:29.259212 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:29.259827 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-43ef1f60-2203-490a-9b68-63caa0f140f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:29.262130 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1462/2909] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:29 2022] DELETE /identity/v3/projects/eeef9069ad924b51a74d509cab103c0a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:29.266984 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196a5ef-2124-45a3-b043-082389569e6f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.267341 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196a5ef-2124-45a3-b043-082389569e6f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.267636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196a5ef-2124-45a3-b043-082389569e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.288675 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8196a5ef-2124-45a3-b043-082389569e6f None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:29.289888 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8196a5ef-2124-45a3-b043-082389569e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:29.295660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:29.350468 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1448/2910] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:29.357912 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945af930-edd7-4850-83c3-e3ae9bff5d64 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.358198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945af930-edd7-4850-83c3-e3ae9bff5d64 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.358475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945af930-edd7-4850-83c3-e3ae9bff5d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.366008 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:29.380697 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-945af930-edd7-4850-83c3-e3ae9bff5d64 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:29.381887 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-945af930-edd7-4850-83c3-e3ae9bff5d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:29.390684 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1463/2911] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:29.394800 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196a5ef-2124-45a3-b043-082389569e6f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.404348 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196a5ef-2124-45a3-b043-082389569e6f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.408742 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.409330 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.409606 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.409875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.410504 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:09:29.410803 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.412385 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.423403 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1449/2912] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 02:09:29.427494 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945af930-edd7-4850-83c3-e3ae9bff5d64 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.437689 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945af930-edd7-4850-83c3-e3ae9bff5d64 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.442490 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.443117 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.443400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.443674 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.444377 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.446298 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.474331 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1464/2913] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:29 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:29.478454 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.488192 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb7de-e5d3-48e6-9ec1-600c048a3ba2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.543088 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.543679 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.543960 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.544236 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.544896 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.546780 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.567623 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1450/2914] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:29 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:29.571731 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.582611 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558e9b8-44cd-44e1-86f3-7a987dc957a2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.587176 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.587764 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.588040 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.588347 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.588982 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.590566 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.600075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1465/2915] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:29.603834 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.613422 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf8f5-5f35-4c00-896d-c319a735a3ae admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.617951 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d251969-3041-4dc9-9d69-b1285e67b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.618548 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251969-3041-4dc9-9d69-b1285e67b832 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.618792 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251969-3041-4dc9-9d69-b1285e67b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.619061 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251969-3041-4dc9-9d69-b1285e67b832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.620189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d251969-3041-4dc9-9d69-b1285e67b832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.621768 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d251969-3041-4dc9-9d69-b1285e67b832 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.631821 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1451/2916] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:29.636396 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.646284 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a43cd-4fb0-4bfb-bbf6-9f26df609a52 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.650879 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.651553 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.651850 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.652122 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] PATH_INFO: `/v3/projects/1f882840a1474cc6beeaeddee226704f/users/46d7d5fb1e854787b374b7bbd8807fb1/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.666020 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f882840a1474cc6beeaeddee226704f, user_id=46d7d5fb1e854787b374b7bbd8807fb1, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.669197 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.681514 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1466/2917] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:29 2022] PUT /identity/v3/projects/1f882840a1474cc6beeaeddee226704f/users/46d7d5fb1e854787b374b7bbd8807fb1/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 02:09:29.692279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6092df0-6b98-4cc8-8ae2-2587d4b09373 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.692564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6092df0-6b98-4cc8-8ae2-2587d4b09373 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.692860 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6092df0-6b98-4cc8-8ae2-2587d4b09373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.719303 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d6092df0-6b98-4cc8-8ae2-2587d4b09373 None None] MFA Rules not processed for user `46d7d5fb1e854787b374b7bbd8807fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:29.720517 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6092df0-6b98-4cc8-8ae2-2587d4b09373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:29.782355 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1452/2918] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:29.793367 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:29.795069 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:29.849833 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425a64f-6ae3-4514-9cc1-6a59965ba678 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:29.853446 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a1f2-4611-4156-baaa-500ccc024a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:29.854593 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a1f2-4611-4156-baaa-500ccc024a76 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:29.854805 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a1f2-4611-4156-baaa-500ccc024a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:29.855078 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a1f2-4611-4156-baaa-500ccc024a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:29.863166 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a1f2-4611-4156-baaa-500ccc024a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:29.864688 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11a1f2-4611-4156-baaa-500ccc024a76 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:29.873927 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1467/2919] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:32.862137 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6092df0-6b98-4cc8-8ae2-2587d4b09373 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:32.871159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:32.923692 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6092df0-6b98-4cc8-8ae2-2587d4b09373 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:32.928112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:32.928750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:32.929027 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:32.929297 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] PATH_INFO: `/v3/users/46d7d5fb1e854787b374b7bbd8807fb1` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:32.930382 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46d7d5fb1e854787b374b7bbd8807fb1)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:32.932336 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:32.977799 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46d7d5fb1e854787b374b7bbd8807fb1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:32.989967 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46d7d5fb1e854787b374b7bbd8807fb1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:32.997252 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46d7d5fb1e854787b374b7bbd8807fb1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:32.997927 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46d7d5fb1e854787b374b7bbd8807fb1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:32.998527 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46d7d5fb1e854787b374b7bbd8807fb1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:33.010991 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1453/2920] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:09:32 2022] DELETE /identity/v3/users/46d7d5fb1e854787b374b7bbd8807fb1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:33.015680 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.016182 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.016322 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.035018 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:33.036201 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:33.093179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1468/2921] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:33.102617 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.103986 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:33.159905 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653389e-aaf6-47ab-9bd0-1c09042f6827 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.163603 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242ac8-d542-487a-83c7-14591b9f703e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.164197 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242ac8-d542-487a-83c7-14591b9f703e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.164505 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242ac8-d542-487a-83c7-14591b9f703e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.164778 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242ac8-d542-487a-83c7-14591b9f703e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.174159 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242ac8-d542-487a-83c7-14591b9f703e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.175576 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242ac8-d542-487a-83c7-14591b9f703e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.184795 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1454/2922] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:33.449378 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.450803 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:33.465694 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c28f9d-0af5-4888-83bd-17e4ce50dfb8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.470686 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.471836 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.472420 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.472770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] PATH_INFO: `/v3/projects/1f882840a1474cc6beeaeddee226704f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.474159 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f882840a1474cc6beeaeddee226704f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.476144 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.513245 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:33.523180 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f882840a1474cc6beeaeddee226704f', 'request_id': 'req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:33.529595 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f882840a1474cc6beeaeddee226704f', 'request_id': 'req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:33.535108 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f882840a1474cc6beeaeddee226704f', 'request_id': 'req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:33.547109 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] The token cache is being invalidate because project 1f882840a1474cc6beeaeddee226704f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:09:33.547429 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:33.548058 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-cab9543b-a7ee-47e4-9e12-3d7cdc066f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:09:33.549594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1469/2923] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:09:33 2022] DELETE /identity/v3/projects/1f882840a1474cc6beeaeddee226704f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:33.554469 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff616-67a5-457c-a7d4-03f87df1419a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.554782 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff616-67a5-457c-a7d4-03f87df1419a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.555056 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff616-67a5-457c-a7d4-03f87df1419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.574629 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4fbff616-67a5-457c-a7d4-03f87df1419a None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:33.575816 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4fbff616-67a5-457c-a7d4-03f87df1419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:33.599564 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:33.633985 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1455/2924] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:33.642404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61bda1-70de-471f-a19d-cb639e546b70 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.642693 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61bda1-70de-471f-a19d-cb639e546b70 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.642971 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61bda1-70de-471f-a19d-cb639e546b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.662973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6f61bda1-70de-471f-a19d-cb639e546b70 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:33.664170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6f61bda1-70de-471f-a19d-cb639e546b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:33.672973 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1470/2925] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:09:33.677259 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff616-67a5-457c-a7d4-03f87df1419a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.686939 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff616-67a5-457c-a7d4-03f87df1419a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.691493 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.692080 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.692392 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.692668 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.693298 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 02:09:33.693598 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.695281 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.706831 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1456/2926] 10.222.0.50 () {58 vars in 1153 bytes} [Tue Aug 30 02:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 02:09:33.711149 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61bda1-70de-471f-a19d-cb639e546b70 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.721344 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61bda1-70de-471f-a19d-cb639e546b70 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.726320 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.726914 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.727191 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.727462 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.728097 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.730068 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.756655 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1471/2927] 10.222.0.50 () {60 vars in 1153 bytes} [Tue Aug 30 02:09:33 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 02:09:33.761262 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.771420 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3c826-2b0e-484f-a16b-1649b6981e93 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.825728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.826316 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.826589 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.826856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.827473 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.829381 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.849614 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1457/2928] 10.222.0.50 () {60 vars in 1144 bytes} [Tue Aug 30 02:09:33 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 02:09:33.854328 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.864786 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4bfc4-08ad-4fc1-b37f-f45ec3b8cc6a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.869373 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.870451 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.870724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.870990 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.871626 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.873239 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.882873 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1472/2929] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:33.887807 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.897630 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3620ed-8e6e-45bd-94c3-d9a0ba2fa9f4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.902251 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d2ac7-9cce-426f-b0d3-6f6a708703ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.902841 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2ac7-9cce-426f-b0d3-6f6a708703ab admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.903120 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2ac7-9cce-426f-b0d3-6f6a708703ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.903396 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d2ac7-9cce-426f-b0d3-6f6a708703ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.904036 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d2ac7-9cce-426f-b0d3-6f6a708703ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.905655 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d2ac7-9cce-426f-b0d3-6f6a708703ab admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.916934 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1458/2930] 10.222.0.50 () {58 vars in 1122 bytes} [Tue Aug 30 02:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 02:09:33.920959 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:33.930671 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a5352-b1b5-4e10-ae93-e29ca6cf8574 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:33.935782 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:33.936452 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.936734 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.937010 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] PATH_INFO: `/v3/projects/2b96716785144cbea5a2fd63c2afeec1/users/07b2c492f3c74244afe66a907c8948c8/roles/61a194f6aba541de8c32f53dfc572624` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:33.951696 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b96716785144cbea5a2fd63c2afeec1, user_id=07b2c492f3c74244afe66a907c8948c8, role_id=61a194f6aba541de8c32f53dfc572624)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:33.954350 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:33.967937 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1473/2931] 10.222.0.50 () {60 vars in 1483 bytes} [Tue Aug 30 02:09:33 2022] PUT /identity/v3/projects/2b96716785144cbea5a2fd63c2afeec1/users/07b2c492f3c74244afe66a907c8948c8/roles/61a194f6aba541de8c32f53dfc572624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:09:33.978310 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565febb-37c0-440d-9b26-f57e960de209 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:33.978605 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565febb-37c0-440d-9b26-f57e960de209 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:33.978887 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565febb-37c0-440d-9b26-f57e960de209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:34.006206 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f565febb-37c0-440d-9b26-f57e960de209 None None] MFA Rules not processed for user `07b2c492f3c74244afe66a907c8948c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:09:34.007408 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f565febb-37c0-440d-9b26-f57e960de209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:34.069166 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1459/2932] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:09:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 02:09:34.080747 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:09:34.082404 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:09:34.138261 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff463b-e242-4dcc-a128-5898ad1f6194 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:09:34.141895 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17316ab7-e7d9-4fb3-8837-2d48883978b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b68fc66dbef44519a2212d35655bf6a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:09:34.142521 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316ab7-e7d9-4fb3-8837-2d48883978b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:09:34.142824 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316ab7-e7d9-4fb3-8837-2d48883978b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:09:34.143118 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17316ab7-e7d9-4fb3-8837-2d48883978b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:09:34.151170 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316ab7-e7d9-4fb3-8837-2d48883978b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:09:34.152678 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17316ab7-e7d9-4fb3-8837-2d48883978b3 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:09:34.161195 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1474/2933] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:09:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:09:44.066346 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:09:44.157272 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:10:19.641126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565febb-37c0-440d-9b26-f57e960de209 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:10:19.700810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565febb-37c0-440d-9b26-f57e960de209 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:10:19.705307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:10:19.705941 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:10:19.706224 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:10:19.706526 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] PATH_INFO: `/v3/users/07b2c492f3c74244afe66a907c8948c8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:10:19.707693 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b2c492f3c74244afe66a907c8948c8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:10:19.709688 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:10:19.753638 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b2c492f3c74244afe66a907c8948c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:19.765961 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b2c492f3c74244afe66a907c8948c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:19.772636 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b2c492f3c74244afe66a907c8948c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:19.773283 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b2c492f3c74244afe66a907c8948c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:19.773883 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b2c492f3c74244afe66a907c8948c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:19.786812 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1460/2934] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:10:19 2022] DELETE /identity/v3/users/07b2c492f3c74244afe66a907c8948c8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:10:19.791317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:10:19.791806 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:10:19.791924 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:10:19.810727 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] MFA Rules not processed for user `a8eb78c86bca426c9163deb9994b5cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 02:10:19.811900 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:10:19.868738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1475/2935] 10.222.0.50 () {58 vars in 958 bytes} [Tue Aug 30 02:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 02:10:19.878229 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:10:19.879646 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:10:19.941234 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92474597-a9a5-4720-b625-3733f166afe5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:10:19.944867 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:10:19.945467 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:10:19.945744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:10:19.946018 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:10:19.955410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:10:19.956856 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:10:19.966708 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1461/2936] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:10:20.227754 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:10:20.229198 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:10:20.241785 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8c32c-5c5b-4099-a664-cede72a35637 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:10:20.246839 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:10:20.247475 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:10:20.247770 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:10:20.248028 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] PATH_INFO: `/v3/projects/2b96716785144cbea5a2fd63c2afeec1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:10:20.249237 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b96716785144cbea5a2fd63c2afeec1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:10:20.251247 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:10:20.298883 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b96716785144cbea5a2fd63c2afeec1', 'request_id': 'req-3349faeb-7182-4d14-a2ce-e552056e6a36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:20.304609 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b96716785144cbea5a2fd63c2afeec1', 'request_id': 'req-3349faeb-7182-4d14-a2ce-e552056e6a36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:20.311007 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b96716785144cbea5a2fd63c2afeec1', 'request_id': 'req-3349faeb-7182-4d14-a2ce-e552056e6a36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:20.322564 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] The token cache is being invalidate because project 2b96716785144cbea5a2fd63c2afeec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:10:20.322851 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:20.323472 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:10:20.325016 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1476/2937] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:10:20 2022] DELETE /identity/v3/projects/2b96716785144cbea5a2fd63c2afeec1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:10:29.961969 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:10:30.319539 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:10:55.805198 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:10:55.814716 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d90027-447c-4439-8d75-c5b2a277bd39 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:10:55.868450 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:10:55.869060 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:10:55.869340 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:10:55.869615 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:10:55.871410 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:10:55.941307 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:10:55.942710 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:10:55.953925 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1462/2938] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:10:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:10:57.132494 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:10:57.133968 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:10:57.146864 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349faeb-7182-4d14-a2ce-e552056e6a36 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:10:57.150456 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:10:57.151051 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:10:57.151329 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:10:57.151602 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:10:57.153258 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:10:57.166317 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:10:57.167738 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:10:57.179067 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1477/2939] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:10:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:11:05.948660 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=54 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:11:06.913645 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:11:06.922578 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32114b61-eb71-4116-822e-fce268057d95 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:11:06.926188 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1c3b7f7c83f48aca27fbd46202f6506', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:11:06.926772 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:11:06.927048 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:11:06.927317 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:11:06.929073 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:11:06.991728 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:11:06.993180 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:11:07.003282 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1463/2940] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:11:07.174294 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:11:10.011155 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:11:10.012926 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:11:10.084594 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a2fe7-4ac0-420b-8972-774f24fa3a6b service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:11:10.090890 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:11:10.091512 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:11:10.091787 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:11:10.092060 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:11:10.093939 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:11:10.106894 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:11:10.108561 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:11:10.118994 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1478/2941] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:11:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:11:13.191437 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:11:13.192875 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:11:13.205750 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de39390-408c-4fed-bb27-b7e665a2bd1c service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:11:13.210391 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:11:13.210957 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:11:13.211232 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:11:13.211507 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] PATH_INFO: `/v3/users/bd08361520db465bb9069326d03d0307` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:11:13.225490 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd08361520db465bb9069326d03d0307)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:11:13.227432 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:11:13.271279 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd08361520db465bb9069326d03d0307'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.283460 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd08361520db465bb9069326d03d0307'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.290546 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd08361520db465bb9069326d03d0307'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.291273 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd08361520db465bb9069326d03d0307'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.291898 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd08361520db465bb9069326d03d0307'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.306126 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1464/2942] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:11:13 2022] DELETE /identity/v3/users/bd08361520db465bb9069326d03d0307 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:11:13.474400 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:11:13.486672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bae674-12e4-4c4f-a4ae-76887196dd38 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:11:13.554390 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:11:13.555032 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:11:13.555310 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:11:13.555582 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] PATH_INFO: `/v3/projects/74610e8b7b4d4fafa0e49ba73b94aa48` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:11:13.567205 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=74610e8b7b4d4fafa0e49ba73b94aa48)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:11:13.569179 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:11:13.616538 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74610e8b7b4d4fafa0e49ba73b94aa48', 'request_id': 'req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.623543 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74610e8b7b4d4fafa0e49ba73b94aa48', 'request_id': 'req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.629076 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74610e8b7b4d4fafa0e49ba73b94aa48', 'request_id': 'req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.641995 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] The token cache is being invalidate because project 74610e8b7b4d4fafa0e49ba73b94aa48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:11:13.642280 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.642907 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.644421 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1479/2943] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:11:13 2022] DELETE /identity/v3/projects/74610e8b7b4d4fafa0e49ba73b94aa48 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:11:13.648508 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:11:13.649899 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:11:13.663810 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d5581-0195-4ae5-80a5-c2d95b843062 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:11:13.715784 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:11:13.724226 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:11:13.724980 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:11:13.725464 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:11:13.725926 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] PATH_INFO: `/v3/users/debecc8a5c6b4de08aa0d5c8dea91362` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:11:13.739112 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] RBAC: Authorizing `identity:delete_user(user_id=debecc8a5c6b4de08aa0d5c8dea91362)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:11:13.741068 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:11:13.784847 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debecc8a5c6b4de08aa0d5c8dea91362'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.796964 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debecc8a5c6b4de08aa0d5c8dea91362'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.803594 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debecc8a5c6b4de08aa0d5c8dea91362'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.804264 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debecc8a5c6b4de08aa0d5c8dea91362'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.804874 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debecc8a5c6b4de08aa0d5c8dea91362'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:13.817736 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1465/2944] 10.222.0.50 () {58 vars in 1224 bytes} [Tue Aug 30 02:11:13 2022] DELETE /identity/v3/users/debecc8a5c6b4de08aa0d5c8dea91362 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:11:13.983494 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:11:13.984957 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:11:14.000053 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a9ab1-dc86-45f1-bcb5-b6feff00a50c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:11:14.008184 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:11:14.057589 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eb78c86bca426c9163deb9994b5cec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7c73461b5344be681351a70b1beafa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:11:14.058213 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:11:14.058513 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:11:14.058910 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] PATH_INFO: `/v3/projects/d75ab1367ec047f996b8063a07f68220` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:11:14.069217 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75ab1367ec047f996b8063a07f68220)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:11:14.071123 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:11:14.115469 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75ab1367ec047f996b8063a07f68220', 'request_id': 'req-941836ce-8f61-4c40-83c7-77945b7b4f26'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:14.121888 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75ab1367ec047f996b8063a07f68220', 'request_id': 'req-941836ce-8f61-4c40-83c7-77945b7b4f26'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:14.127392 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75ab1367ec047f996b8063a07f68220', 'request_id': 'req-941836ce-8f61-4c40-83c7-77945b7b4f26'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:14.139793 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] The token cache is being invalidate because project d75ab1367ec047f996b8063a07f68220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 02:11:14.140075 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:14.140724 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-941836ce-8f61-4c40-83c7-77945b7b4f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 02:11:14.142214 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1480/2945] 10.222.0.50 () {58 vars in 1233 bytes} [Tue Aug 30 02:11:13 2022] DELETE /identity/v3/projects/d75ab1367ec047f996b8063a07f68220 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 02:11:23.814451 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:11:24.136672 dv-cinder-819668-6-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 02:13:48.710681 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 02:13:48.712702 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:13:48.851744 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2ccc8-240b-4d11-b3cb-557ff366040f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 02:13:48.908884 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf5e36-f1ba-4205-bc51-19b9d6bf8bce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '588e9ed6789e46d6a6693cb54e6fdce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eba2b9510c624752808c81bcd11a5f54', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 02:13:48.909485 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5e36-f1ba-4205-bc51-19b9d6bf8bce service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 02:13:48.909766 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5e36-f1ba-4205-bc51-19b9d6bf8bce service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 02:13:48.910037 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5e36-f1ba-4205-bc51-19b9d6bf8bce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 02:13:48.911756 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bdaf5e36-f1ba-4205-bc51-19b9d6bf8bce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 02:13:48.922651 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf5e36-f1ba-4205-bc51-19b9d6bf8bce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 02:13:48.924209 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf5e36-f1ba-4205-bc51-19b9d6bf8bce service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 02:13:48.934555 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1466/2946] 10.222.0.50 () {60 vars in 1365 bytes} [Tue Aug 30 02:13:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 02:13:58.930189 dv-cinder-819668-6-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}